Machine Learning for Detecting Data Exfiltration: A Review

Abstract

Research at the intersection of cybersecurity, Machine Learning (ML), and Software Engineering (SE) has recently taken significant steps in proposing countermeasures for detecting sophisticated data exfiltration attacks. It is important to systematically review and synthesize the ML-based data exfiltration countermeasures for building a body of knowledge on this important topic. Objective: This paper aims at systematically reviewing ML-based data exfiltration countermeasures to identify and classify ML approaches, feature engineering techniques, evaluation datasets, and performance metrics used for these countermeasures. This review also aims at identifying gaps in research on ML-based data exfiltration countermeasures. Method: We used Systematic Literature Review (SLR) method to select and review 92 papers. Results: The review has enabled us to (a) classify the ML approaches used in the countermeasures into data-driven, and behaviour-driven approaches, (b) categorize features into six types: behavioral, content-based, statistical, syntactical, spatial and temporal, (c) classify the evaluation datasets into simulated, synthesized, and real datasets and (d) identify 11 performance measures used by these studies. Conclusion: We conclude that: (i) the integration of data-driven and behaviour-driven approaches should be explored; (ii) There is a need of developing high quality and large size evaluation datasets; (iii) Incremental ML model training should be incorporated in countermeasures; (iv) resilience to adversarial learning should be considered and explored during the development of countermeasures to avoid poisoning attacks; and (v) the use of automated feature engineering should be encouraged for efficiently detecting data exfiltration attacks.

SmartValidator: A Framework for Automatic Identification and Classification of Cyber Threat Data

“…Preprocessed data is passed to a feature engineering module where data is transformed into features (numeric form) (step 2, Figure 5), which are used as input for the ML algorithms. The reason behind this is that ML algorithms can only work with numerical data [2,39,40,50]. Depending on the type, size and diversity of CTI, the data science team chooses a feature engineering approach.…”

“…Depending on the nature of the training data, different models are built by a data science team. Traditionally a set of ML algorithms are applied to find an algorithm suitable for a specific dataset and user requirements [1,2,40]. To investigate the effectiveness of prediction models for the validation task, we considered a set of ML algorithms (e.g., Decision Tree, Naïve Bayes, K-Nearest Neighbours and Random Forest).…”

“…The built model was optimised by performing hyperparameter tuning. The Python module sci-kit learn was used to build the prediction models, as it is one of the popular and widely used libraries for building prediction models [10,29,40].…”

“…This will determine the effectiveness and efficiency of SmartValidator. Accuracy, precision, recall and F1-score are the four commonly accepted evaluation metrics for evaluating a prediction models performance [10,40]. The correct and incorrect predictions are further calculated using number of (i)True Positive (TP) (refers to correct prediction of an attributes label), (ii) False Positive (FP) (indicates incorrect prediction of an attributes label), (iii) True Negative (TN) (refers to correct prediction that a threat does not have a particular label) and (iv) False Negative (FN) (indicates incorrect prediction that a threat does not have particular label).…”

A Review on C3I Systems' Security: Vulnerabilities, Attacks, and Countermeasures

“…Hence, it is evident that researchers have had a narrow focus on the use of AI/ML-based techniques to enhance the security of C3I systems. By studying contemporary literature and their future directions, we want to draw researchers and practitioners attention to explore and incorporate AI/ML-enabled technologies such as the use of ML for detecting data exfiltration [81], AI for national security [82], and AI/ML-enabled cybersecurity [83], [84].…”