Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning

Liu, Lan; Wang, Pengcheng; Lin, Jun

doi:10.1109/access.2020.3048198

Cited by 158 publications

(59 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training classifiers on imbalanced datasets can affect their performance, both due to the imbalanced ratio of attack to benign traffic and the imbalance between several attack classes. Some methods have been proposed to augment or synthetically inflate minority samples for attack traffic [29]. As an anomaly-detection method, CBAM is, however, trained on a self-supervised way strictly on benign traffic, with no attack traffic being present in the training data.…”

Section: Sample Imbalance and Evaluation Methodologymentioning

confidence: 99%

See 1 more Smart Citation

CBAM: A Contextual Model for Network Anomaly Detection

2021

View full text Add to dashboard Cite

Anomaly-based intrusion detection methods aim to combat the increasing rate of zero-day attacks, however, their success is currently restricted to the detection of high-volume attacks using aggregated traffic features. Recent evaluations show that the current anomaly-based network intrusion detection methods fail to reliably detect remote access attacks. These are smaller in volume and often only stand out when compared to their surroundings. Currently, anomaly methods try to detect access attack events mainly as point anomalies and neglect the context they appear in. We present and examine a contextual bidirectional anomaly model (CBAM) based on deep LSTM-networks that is specifically designed to detect such attacks as contextual network anomalies. The model efficiently learns short-term sequential patterns in network flows as conditional event probabilities. Access attacks frequently break these patterns when exploiting vulnerabilities, and can thus be detected as contextual anomalies. We evaluated CBAM on an assembly of three datasets that provide both representative network access attacks, real-life traffic over a long timespan, and traffic from a real-world red-team attack. We contend that this assembly is closer to a potential deployment environment than current NIDS benchmark datasets. We show that, by building a deep model, we are able to reduce the false positive rate to 0.16% while effectively detecting six out of seven access attacks, which is significantly lower than the operational range of other methods. We further demonstrate that short-term flow structures remain stable over long periods of time, making the CBAM robust against concept drift.

show abstract

Section: Sample Imbalance and Evaluation Methodologymentioning

confidence: 99%

“…Zhou et al [38] used embeddings to facilitate anomaly-detection that is robust against dataset imbalances. Liu et al [29] use embeddings to augment and inflate minority class data samples for the same purpose.…”

Section: Related Workmentioning

confidence: 99%

CBAM: A Contextual Model for Network Anomaly Detection

2021

View full text Add to dashboard Cite

show abstract

“…The results showed that random forest (RF) outperformed deep learning algorithms. Two imbalanced datasets of similar size to the above reference were used in [51] to test a resampling technique proposed in the work with different machine learning algorithms including RF and LSTM. The proposal was compared with popular resampling techniques such as random undersampling (RUS), random oversampling (ROS) or SMOTE.…”

Section: Related Workmentioning

confidence: 99%

Selecting the Suitable Resampling Strategy for Imbalanced Data Classification Regarding Dataset Properties. An Approach Based on Association Models

2021

View full text Add to dashboard Cite

In many application domains such as medicine, information retrieval, cybersecurity, social media, etc., datasets used for inducing classification models often have an unequal distribution of the instances of each class. This situation, known as imbalanced data classification, causes low predictive performance for the minority class examples. Thus, the prediction model is unreliable although the overall model accuracy can be acceptable. Oversampling and undersampling techniques are well-known strategies to deal with this problem by balancing the number of examples of each class. However, their effectiveness depends on several factors mainly related to data intrinsic characteristics, such as imbalance ratio, dataset size and dimensionality, overlapping between classes or borderline examples. In this work, the impact of these factors is analyzed through a comprehensive comparative study involving 40 datasets from different application areas. The objective is to obtain models for automatic selection of the best resampling strategy for any dataset based on its characteristics. These models allow us to check several factors simultaneously considering a wide range of values since they are induced from very varied datasets that cover a broad spectrum of conditions. This differs from most studies that focus on the individual analysis of the characteristics or cover a small range of values. In addition, the study encompasses both basic and advanced resampling strategies that are evaluated by means of eight different performance metrics, including new measures specifically designed for imbalanced data classification. The general nature of the proposal allows the choice of the most appropriate method regardless of the domain, avoiding the search for special purpose techniques that could be valid for the target data.

show abstract

“…In addition, new device types and network structures have emerged, such as 5G, cloud computing, and the Internet of Things (IoT). With the growth of these systems and networks, ensuring their cyber security is critical [14]. According to the 2021 Cyber Threat Defense Report (CyberEdge, 2021) [4] released by the CyberEdge team and compiled from data provided by 1200 IT security experts working in 19 different sectors in 17 different countries, attacks to companies connected to the internet network have been increased (Figure -1).…”

Section: Introductionmentioning

confidence: 99%

FastTrafficAnalyzer: An Efficient Method for Intrusion Detection Systems to Analyze Network Traffic

Arslan¹

2021

DÜMF Mühendislik Dergisi

View full text Add to dashboard Cite

Network intrusion detection systems are software or devices used to detect malignant attackers in modern internet networks. The success of these systems depends on the performance of the algorithm and method used to catch attacks and the time it takes for it. Due to the continuous internet traffic, these systems are expected to detect attacks in real time. In this study, using a proposed pre-processing, internet traffic data becomes more easily processable and traffic is classified by network analysis with machine learning techniques. In this way, the traffic analysis time was significantly shortened and a high level of success was achieved. The proposed model has been tested in the CSE-CIC-IDS2018 dataset and its advantaged verified. Experimental results i) 99.0% detection rate was achieved in the ExtraTree algorithm for binary classification, while a reduction of 82.96% was achieved in the processing time per sample; ii) For multiclass (15 class) detection, 98.5% detection rate was achieved with the Random Forest algorithm, while a 64.43% shortening was achieved in the processing time per sample. As a result, similar classification rate with the studies in the literature has been achieved with much shorter test time.

show abstract

Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning

Cited by 158 publications

References 38 publications

CBAM: A Contextual Model for Network Anomaly Detection

CBAM: A Contextual Model for Network Anomaly Detection

Selecting the Suitable Resampling Strategy for Imbalanced Data Classification Regarding Dataset Properties. An Approach Based on Association Models

FastTrafficAnalyzer: An Efficient Method for Intrusion Detection Systems to Analyze Network Traffic

Contact Info

Product

Resources

About