Big Data Analytics for Intrusion Detection System: Statistical Decision-Making Using Finite Dirichlet Mixture Models

Moustafa, Nour; Creech, Gideon; Slay, Jill

doi:10.1007/978-3-319-59439-2_5

Cited by 131 publications

(72 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results presented in table 2 show that the random forest Algorithm performed well in terms of overall accuracy and model precision. Though, it shows a [38] 92.29% 92.2% 0.41% -DMM [39] 97.8% 97.8% 2.5% -TANN [40] 96.91% 97.8% 2.5% -DBN [41] 97.45% --3.2 sec RNN [42] 99.53% 97.09% 3.6% 5516 sec DNN [43] 75.75% 75% 15% -E-DNN [44] 92.49% 98% 14.7% -DFF-NN [45] 98.6% 99% 1.8% 398 sec DL [46] 98% 71% --SVM-DR [46] 97 very low precision for U2R and R2L attacks. J48 detects attacks with very good accuracy and low missclassification rate (or CPE).…”

Section: ) Results Discussionmentioning

confidence: 95%

A Machine Learning Security Framework for Iot Systems

et al. 2020

View full text Add to dashboard Cite

Internet of Things security is attracting a growing attention from both academic and industry communities. Indeed, IoT devices are prone to various security attacks varying from Denial of Service (DoS) to network intrusion and data leakage. This paper presents a novel machine learning (ML) based security framework that automatically copes with the expanding security aspects related to IoT domain. This framework leverages both Software Defined Networking (SDN) and Network Function Virtualization (NFV) enablers for mitigating different threats. This AI framework combines monitoring agent and AIbased reaction agent that use ML-Models divided into network patterns analysis, along with anomalybased intrusion detection in IoT systems. The framework exploits the supervised learning, distributed data mining system and neural network for achieving its goals. Experiments results demonstrate the efficiency of the proposed scheme. In particular, the distribution of the attacks using the data mining approach is highly successful in detecting the attacks with high performance and low cost. Regarding our anomalybased intrusion detection system (IDS) for IoT, we have evaluated the experiment in a real Smart building scenario using one-class SVM. The detection accuracy of anomalies achieved 99.71%. A feasibility study is conducted to identify the current potential solutions to be adopted and to promote the research towards the open challenges.

show abstract

Section: ) Results Discussionmentioning

confidence: 95%

A Machine Learning Security Framework for Iot Systems

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The method in [ 40 ] is based on a feed-forward neural network, with stochastic gradient descent back-propagation for training, yet with the addition of a deep auto-encoder to reduce dimensionality. The results are excellent when compared to the mixture approach presented in [ 31 ], and support vector machines or other deep learning approaches. However, this method uses all features of the data initially, and requires the complexity of neural networks and back-propagation.…”

Section: Introductionmentioning

confidence: 84%

“…Specifically in the context of computer security and intrusion detection, several recent works used the data set given in [ 30 ] (UNSW-NB15) together with novel techniques for parameter estimation. In [ 31 ], it was used successfully for intrusion detection. The decision engine in [ 31 ] assumes a Dirichlet mixture model for the multi-dimensional data (above 40 features per entry in the data set), and it is based on estimating its parameters at the learning phase.…”

Section: Introductionmentioning

confidence: 99%

“…In [ 31 ], it was used successfully for intrusion detection. The decision engine in [ 31 ] assumes a Dirichlet mixture model for the multi-dimensional data (above 40 features per entry in the data set), and it is based on estimating its parameters at the learning phase. The suggested mixture model efficiently estimates the complex relations between the features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Anomaly Detection for Individual Sequences with Applications in Identifying Malicious Tools

Siboni

Cohen

2020

Entropy

View full text Add to dashboard Cite

Anomaly detection refers to the problem of identifying abnormal behaviour within a set of measurements. In many cases, one has some statistical model for normal data, and wishes to identify whether new data fit the model or not. However, in others, while there are normal data to learn from, there is no statistical model for this data, and there is no structured parameter set to estimate. Thus, one is forced to assume an individual sequences setup, where there is no given model or any guarantee that such a model exists. In this work, we propose a universal anomaly detection algorithm for one-dimensional time series that is able to learn the normal behaviour of systems and alert for abnormalities, without assuming anything on the normal data, or anything on the anomalies. The suggested method utilizes new information measures that were derived from the Lempel–Ziv (LZ) compression algorithm in order to optimally and efficiently learn the normal behaviour (during learning), and then estimate the likelihood of new data (during operation) and classify it accordingly. We apply the algorithm to key problems in computer security, as well as a benchmark anomaly detection data set, all using simple, single-feature time-indexed data. The first is detecting Botnets Command and Control (C&C) channels without deep inspection. We then apply it to the problems of malicious tools detection via system calls monitoring and data leakage identification.We conclude with the New York City (NYC) taxi data. Finally, while using information theoretic tools, we show that an attacker’s attempt to maliciously fool the detection system by trying to generate normal data is bound to fail, either due to a high probability of error or because of the need for huge amounts of resources.

show abstract

“…For statistical learning, [15] suggests that using GMM to fit the dataset and identify any outliers may be an alternative option if the Gaussian distribution is not applicable for the data. And as [16] states that GMM can well define all the possible data points by assigning the probability rather than a cluster in the k-means algorithm.…”

Section: Gaussian Mixture Model (Gmm)mentioning

confidence: 99%

Cross-Site Scripting (XSS) Detection Integrating Evidences in Multiple Stages

Zhang¹,

Jou²,

2019

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

As Cross-Site Scripting (XSS) remains one of the top web security risks, people keep exploring ways to detect such attacks efficiently. So far, existing solutions only focus on the payload in a web request or a response, a single stage of a web transaction. This work proposes a new approach that integrates evidences from both a web request and its response in order to better characterize XSS attacks and separate them from normal web transactions. We first collect complete payloads of XSS and normal web transactions from two databases and extract features from them using the Word2vec technique. Next, we train two Gaussian mixture models (GMM) with these features, one for XSS transaction and one for normal web transactions. These two models can generate two probability scores for a new web transaction, which indicate how similar this web transaction is to XSS and normal traffics respectively. Finally, we put together these two GMM models in classification by combining these two probabilities to further improve detection accuracy.

show abstract

Big Data Analytics for Intrusion Detection System: Statistical Decision-Making Using Finite Dirichlet Mixture Models

Cited by 131 publications

References 45 publications

A Machine Learning Security Framework for Iot Systems

A Machine Learning Security Framework for Iot Systems

Anomaly Detection for Individual Sequences with Applications in Identifying Malicious Tools

Cross-Site Scripting (XSS) Detection Integrating Evidences in Multiple Stages

Contact Info

Product

Resources

About