Machine learning for encrypted malicious traffic detection: Approaches, datasets and comparative study

Wang, Zihao; Fok, Kar Wai; Thing, Vrizlynn L. L.

doi:10.1016/j.cose.2021.102542

Cited by 75 publications

(24 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditionally, there are two types of features, i.e., packetlevel features and flow level features [43]. The packet-level features (e.g., payload size, packet size, and payload ratio) are often used to find application anomalies, but they are not effective in providing a clear distinction in values between malicious and legitimate traffic flows, which might affect the detection accuracy.…”

Section: Mpls/ Internetmentioning

confidence: 99%

“…As a promising security method, machine learning based malicious traffic detection algorithms have been proposed as complements of the traditional fixed rule based methods [43], [10]. Table I shows a the comparison between the machine learning based anomaly detection methods and the rule based anomaly detection methods.…”

Section: B Motivationmentioning

confidence: 99%

“…Machine learning technologies are now becoming a new powerful tool in the network anomaly detection field. According to model complexity, we can divide the machine learning based anomaly detection methods into two categories, i.e., traditional machine learning methods, and the deep learning methods [43]. Traditional machine learning technologies based anomaly detection methods can be further divided into two types, i.e., supervised learning and unsupervised learning, where the supervised learning methods always train the detection models with labeled traffic and the unsupervised methods will separate the unlabeled traffic into clusters.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Zhang

et al. 2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

Software Defined Network (SDN) has been widely used in modern network architecture. The SD-WAN is considered as a technology that has a potential to revolutionize the WAN service usage by utilizing the SDN philosophy. Attacking SDN router and controller can affect the network and block the entire services. In this paper, we propose a machine learning based anomalous traffic detection framework named OADSD over SD-WAN that can achieve task independent and has the ability of adapting to the environment. The OADSD adopts Distributed Dynamic Feature Extraction (DDFE) to extract representative features directly from the raw traffic, and proposes the Ondemand Evolving Isolation Forest (OEIF) to make the system adapt to an environment. We provide a theoretical analysis of the performance of the OADSD. We also conduct comprehensive experiments to evaluate the performance of the OADSD with real world public datasets as well as a small real testbed. Our experiments under real world public datasets show that, the OADSD can accurately detect various kinds of attacks with a high performance. Compared with the state-of-the-art systems, the OADSD can achieve up to 60% accuracy improvement.

show abstract

Section: Mpls/ Internetmentioning

confidence: 99%

Section: B Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Zhang

et al. 2023

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

show abstract

“…Therefore, most researchers use binary classification of encrypted traffic, that is, to identify malicious traffic among legitimate traffic. Currently, flow-based machine learning and deep learning methods are the mainstream methods for encrypted traffic classification [6]. Shekhawat et al [3] proposed three machine learning techniques, Random Forest (RF), Support Vector Machine (SVM) and XGBoost to distinguish malicious encrypted traffic from benign encrypted traffic.…”

Section: Related Workmentioning

confidence: 99%

IoT Intrusion Detection System Based on LSTM Model

Li¹,

Chang²

2023

Atlantis Highlights in Computer Sciences

View full text Add to dashboard Cite

Aiming at the problems of time-consuming feature extraction and general efficiency in the detection of en-crypted traffic by traditional machine learning algorithms, an intrusion detection model based on deep learning long short-term memory network (LSTM) was proposed. First, the malicious encrypted traffic in the CTU-13 data set and the normal traffic in the CICIDS-2017 data set are extracted to form a data set; then the binary classification data set processing is completed based on the secure transport layer protocol; finally, the LSTM and one-dimensional convolutional neural networks are trained. Network, two-dimensional convolutional neural network and convolutional neural network-long short-term memory network four deep learning models. The experimental results show that LSTM has significant advantages over the other three models in five evaluation parameters, the accuracy of key parameters is as high as 99.84%, and it performs well in terms of CPU and memory usage, which meets the security requirements of the Internet of Things.

show abstract

“…The mapping relationship is approximated by a specific optimization algorithm to achieve the purpose of predicting the desired result. Compared with the rule-based malicious traffic detection method, machine learning method can better extract the encrypted information, timing relationship and other complex features in network traffic [2]. However, most research related to deep learning implicitly assumes that there is a large amount of data with accurate labels, it's not feasible to obtain such data in the practical production environment for many organizations.…”

Section: Introductionmentioning

confidence: 99%

Malicious Traffic Detection with Class Imbalanced Data Based on Coarse-grained Labels

Li¹,

Liu²,

Wang³

et al. 2022

Proceedings of International Symposium on Grids &Amp; Clouds 2022 — PoS(ISGC2022)

View full text Add to dashboard Cite

In order to resist complex cyber-attacks, a Security Operations Center (SOC) named IHEPSOC has been developed and deployed in the Institute of High Energy Physics (IHEP) of the Chinese Academy of Sciences, which contributed to the reliability and security of the network for IHEP. It has become a major task to integrate state-of-the-art cyber-attack detection methods for IHEPSOC to improve the ability of threat detection. Malicious traffic detection based on machine learning is an emerging security paradigm, which can effectively detect both known and unknown cyberattacks. However, the existing studies usually adopt traditional supervised learning, which often encounter issues when applied to real-world production environment due to its implicit assumptions on the operating dependence. For example, most studies are based on datasets that already have accurate data labels, but labeling these datasets accurately requires significant manual effort. In addition, in the real-world service, the volume of benign traffic data is larger than that of the malicious traffic data, and the imbalance between benign and malicious categories also makes many machine learning detection models difficult to apply to a production environment. Based on these, we propose a detection method for class imbalanced malicious traffic based on coarsegrained data labels, which achieves comparable performance compare to other supervised learning methods. We conducted three experiments, using the Android Malware 2017 dataset, and verified the practicability and effectiveness of the proposed method.

show abstract

Machine learning for encrypted malicious traffic detection: Approaches, datasets and comparative study

Cited by 75 publications

References 54 publications

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

Real-Time Malicious Traffic Detection With Online Isolation Forest Over SD-WAN

IoT Intrusion Detection System Based on LSTM Model

Malicious Traffic Detection with Class Imbalanced Data Based on Coarse-grained Labels

Contact Info

Product

Resources

About