DeepCASE

We provide the DeepCASE class as a wrapper around the ContextBuilder and Interpreter. The DeepCASE class only implements the fit()/predict() methods which requires a priori knowledge of the sequence maliciousness score. If you require a more fine-grained tuning of DeepCASE, e.g., when using the manual labelling mode, we recommend using the individual ContextBuilder and Interpreter objects as shown in the Usage. These individual classes provide a richer API.

class module.DeepCASE(features, max_length=10, hidden_size=128, eps=0.1, min_samples=5, threshold=0.2)[source]
DeepCASE.__init__(features, max_length=10, hidden_size=128, eps=0.1, min_samples=5, threshold=0.2)[source]

Analyse security events with respect to contextual machine behaviour.

Note

When an Interpreter is trained, it heavily depends on the ContextBuilder used during training. Therefore, we strongly suggest not to manually change the context_builder attribute, without retraining the interpreter of the DeepCASE object.

Parameters:

  • features (int) – Number of different possible security events.

  • max_length (int, default=10) – Maximum length of context window as number of events.

  • hidden_size (int, default=128) – Size of hidden layer in sequence to sequence prediction. This parameter determines the complexity of the model and its prediction power. However, high values will result in slower training and prediction times.

  • eps (float, default=0.1) – Epsilon used for determining maximum distance between clusters.

  • min_samples (int, default=5) – Minimum number of required samples per cluster.

  • threshold (float, default=0.2) – Minimum required confidence in fingerprint before using it in training clusters.

Fit/Predict methods

We provide a scikit-learn-like API for DeepCASE to train on sequences with a given maliciousness score and predict the maliciousness score of new sequences.

As DeepCASE is simply a wrapper around the ContextBuilder and Interpreter objects, the following functionality is equivalent:

module.DeepCASE.fit() is equivalent to:

  1. context_builder.ContextBuilder.fit()

  2. interpreter.Interpreter.fit()

module.DeepCASE.predict() is equivalent to:

  1. interpreter.Interpreter.predict()

module.DeepCASE.fit_predict() is equivalent to:

  1. module.DeepCASE.fit()

  2. module.DeepCASE.predict()

Fit

The fit() method provides an API for directly learning the maliciousness score of sequences. This method combines the fit() methods from both the ContextBuilder and the Interpreter. We note that to use the fit() method, scores of sequences should be known a priori. See the interpreter.Interpreter.fit() method for an explanation of how these scores are used.

DeepCASE.fit(X, y, scores, epochs=10, batch_size=128, learning_rate=0.01, optimizer=<class 'torch.optim.sgd.SGD'>, teach_ratio=0.5, iterations=100, query_batch_size=1024, strategy='max', NO_SCORE=-1, verbose=True)[source]

Fit DeepCASE with given data. This method is provided as a wrapper and is equivalent to calling: - context_builder.fit() and - interpreter.fit() in the given order.

Parameters:

  • X (array-like of type=int and shape=(n_samples, context_size)) – Input context to train with.

  • y (array-like of type=int and shape=(n_samples, n_future_events)) – Sequences of target events.

  • scores (array-like of float, shape=(n_samples,)) – Scores for each sample in cluster.

  • epochs (int, default=10) – Number of epochs to train with.

  • batch_size (int, default=128) – Batch size to use for training.

  • learning_rate (float, default=0.01) – Learning rate to use for training.

  • optimizer (optim.Optimizer, default=torch.optim.SGD) – Optimizer to use for training.

  • teach_ratio (float, default=0.5) – Ratio of sequences to train including labels.

  • iterations (int, default=100) – Number of iterations for query.

  • query_batch_size (int, default=1024) – Size of batch for query.

  • strategy (string (max|min|avg), default=max) – Strategy to use for computing scores per cluster based on scores of individual events. Currently available options are: - max: Use maximum score of any individual event in a cluster. - min: Use minimum score of any individual event in a cluster. - avg: Use average score of any individual event in a cluster.

  • NO_SCORE (float, default=-1) – Score to indicate that no score was given to a sample and that the value should be ignored for computing the cluster score. The NO_SCORE value will also be given to samples that do not belong to a cluster.

  • verbose (boolean, default=True) – If True, prints progress.

Returns:

self – Returns self.

Return type:

self

Predict

When DeepCASE is trained, we can use DeepCASE to predict the score of new sequences. To this end, we provide the predict() function which takes context and events as input and outputs the labels of corresponding predicted clusters. If no sequence could be matched, one of the following scores will be given:

  • -1: Not confident enough for prediction

  • -2: Label not in training

  • -3: Closest cluster > epsilon

Note

This method is a wrapper around the interpreter.Interpreter.predict() method.

DeepCASE.predict(X, y, iterations=100, batch_size=1024, verbose=False)[source]

Predict maliciousness of context samples.

Parameters:

  • X (torch.Tensor of shape=(n_samples, seq_length)) – Input context for which to predict maliciousness.

  • y (torch.Tensor of shape=(n_samples, 1)) – Events for which to predict maliciousness.

  • iterations (int, default=100) – Iterations used for optimization.

  • batch_size (int, default=1024) – Batch size used for optimization.

  • verbose (boolean, default=False) – If True, print progress.

Returns:

result – Predicted maliciousness score. Positive scores are maliciousness scores. A score of 0 means we found a match that was not malicious. Special cases:

  • -1: Not confident enough for prediction

  • -2: Label not in training

  • -3: Closest cluster > epsilon

Return type:

np.array of shape=(n_samples,)

Fit_predict

Similar to the scikit-learn API, the fit_predict() method performs the fit() and predict() functions in sequence on the same data.

DeepCASE.fit_predict(X, y, scores, epochs=10, batch_size=128, learning_rate=0.01, optimizer=<class 'torch.optim.sgd.SGD'>, teach_ratio=0.5, iterations=100, query_batch_size=1024, strategy='max', NO_SCORE=-1, verbose=True)[source]

Fit DeepCASE with given data and predict that same data. This method is provided as a wrapper and is equivalent to calling: - self.fit() and - self.predict() in the given order.

Parameters:

  • X (array-like of type=int and shape=(n_samples, context_size)) – Input context to train with.

  • y (array-like of type=int and shape=(n_samples, n_future_events)) – Sequences of target events.

  • scores (array-like of float, shape=(n_samples,)) – Scores for each sample in cluster.

  • epochs (int, default=10) – Number of epochs to train with.

  • batch_size (int, default=128) – Batch size to use for training.

  • learning_rate (float, default=0.01) – Learning rate to use for training.

  • optimizer (optim.Optimizer, default=torch.optim.SGD) – Optimizer to use for training.

  • teach_ratio (float, default=0.5) – Ratio of sequences to train including labels.

  • iterations (int, default=100) – Number of iterations for query.

  • query_batch_size (int, default=1024) – Size of batch for query.

  • strategy (string (max|min|avg), default=max) – Strategy to use for computing scores per cluster based on scores of individual events. Currently available options are: - max: Use maximum score of any individual event in a cluster. - min: Use minimum score of any individual event in a cluster. - avg: Use average score of any individual event in a cluster.

  • NO_SCORE (float, default=-1) – Score to indicate that no score was given to a sample and that the value should be ignored for computing the cluster score. The NO_SCORE value will also be given to samples that do not belong to a cluster.

  • verbose (boolean, default=True) – If True, prints progress.

Returns:

result – Predicted maliciousness score. Positive scores are maliciousness scores. A score of 0 means we found a match that was not malicious. Special cases:

  • -1: Not confident enough for prediction

  • -2: Label not in training

  • -3: Closest cluster > epsilon

Return type:

np.array of shape=(n_samples,)

I/O methods

DeepCASE can be saved and loaded from files using the following methods. Please note that the module.DeepCASE.load() method is a classmethod and must be called statically.

DeepCASE.save(outfile)[source]

Save DeepCASE model to output file.

Parameters:

outfile (string) – Path to output file in which to store DeepCASE model.

classmethod DeepCASE.load(infile, device=None)[source]

Load DeepCASE model from input file.

Parameters:

  • infile (string) – Path to input file from which to load DeepCASE model.

  • device (string, optional) – If given, cast DeepCASE automatically to device.

Example:

from deepcase import DeepCASE
deepcase = DeepCASE.load('<path_to_saved_deepcase_object>')
deepcase.save('<path_to_save_deepcase_object>')