Application of deep autoencoder as an one-class classifier for unsupervised network intrusion detection: a comparative evaluation

Abstract

The ever-increasing use of internet has opened a new avenue for cybercriminals, alarming the online businesses and organization to stay ahead of evolving thread landscape. To this end, intrusion detection system (IDS) is deemed as a promising defensive mechanism to ensure network security. Recently, deep learning has gained ground in the field of intrusion detection but majority of progress has been witnessed on supervised learning which requires adequate labeled data for training. In real practice, labeling the high volume of network traffic is laborious and error prone. Intuitively, unsupervised deep learning approaches has received gaining momentum. Specifically, the advances in deep learning has endowed autoencoder (AE) with greater ability for data reconstruction to learn the robust feature representation from massive amount of data. Notwithstanding, there is no study that evaluates the potential of different AE variants as one-class classifier for intrusion detection. This study fills this gap of knowledge presenting a comparative evaluation of different AE variants for one-class unsupervised intrusion detection. For this research, the evaluation includes five different variants of AE such as Stacked AE, Sparse AE, Denoising AE, Contractive AE and Convolutional AE. Further, the study intents to conduct a fair comparison establishing a unified network configuration and training scheme for all variants over the common benchmark datasets, NSL-KDD and UNSW-NB15. The comparative evaluation study provides a valuable insight on how different AE variants can be used as one-class classifier to build an effective unsupervised IDS. The outcome of this study will be of great interest to the network security community as it provides a promising path for building effective IDS based on deep learning approaches alleviating the need for adequate and diverse intrusion network traffic behavior.

Feature Selection and Ensemble-Based Intrusion Detection System: An Efficient and Comprehensive Approach

“…Regardless of the significant inherent challenge of the NSL-KDD dataset, for instance, the inadequate reflection of current low footprint attack scenarios, it is still considered the most preferred IDSs evaluation dataset because of its distinctive attribute of maximizing predictions for classifiers [15]. It consists of four attack categories with 41 attributes and a single labeled class distinguishing between malicious or regular network traffic [21]. Finally, interested readers can refer to reference [64] for the detailed theoretical and technical documentation of the NSL-KDD dataset.…”

“…Furthermore, irrespective of the phenomenal achievements of researchers and professionals in mitigating the escalating security challenges through IDSs, the challenges of improving the detection rate, accuracy, detection of novel attacks, and reducing false alarm rates are still issues yet to be addressed in the research domain of IDSs. Moreover, the mind-blowing complexity of the present cutting-edge networks has challenged the detection capability of many existing IDSs [21]. Consequently, the past years have seen many researchers and professionals exploit the novelty of machine learning techniques to address the problems mentioned earlier by designing, developing, and deploying effective and efficient IDSs [11].…”

“…Clustering algorithms such as K-means and DBSCAN attract significant research interest to mitigate the shortcomings of processing massive complex unlabeled datasets. Consequently, improving the detection accuracy and substantially reduces the significant false alarm rate of existing IDSs [21]. For example, the authors of [23] used a classical machine learning approach to implement a novel semi-supervised anomaly detection system.…”

SALAD: An Exploration of Split Active Learning based Unsupervised Network Data Stream Anomaly Detection using Autoencoders

“…In our previous work [12], we reviewed autoencoder based anomaly intrusion detection methods, whereby single layer denoising models [13], Long Short Term Memory (LSTM), Recurrent Neural Network [14], [15], ensembled stacked autoencoders [16], [17], and sparsely connected networks [18], [15] were demonstrated across a range of IDS data sets. Vaiyapuri and Binbusayyis [19] evaluated a number of autoencoder network architectures for anomaly detection, finding the use of a contractive penalty to regulate the network provided the best performance when evaluated offline using the NSL-KDD and UNSW-NB15 data sets.…”

“…A number of methods were proposed in the literature to determine the anomaly threshold, an important parameter in deciding whether to label a sample as a positive detection. The threshold can be set to the average RE value observed during training [19]. Naïve Anomaly Threshold (NAT) sets the threshold at the maximum observed RE during training [16].…”