Zero-Day Attack Identification in Streaming Data Using Semantics and Spark

Pallaprolu, Sai C.; Sankineni, Rishi; Thevar, Muthukumar; Karabatis, George; Wang, Jianwu

doi:10.1109/bigdatacongress.2017.25

Cited by 10 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pallaprolu et al [32] proposed that a combination of semantic link networks (SLN) and dynamic semantic graph generation can be used to detect zero-day network intrusion attacks. They used the KDD CUP'99 dataset to evaluate their methodology, and implemented the minimum redundancy maximum relevance (MRMR) feature selection method.…”

Section: A Network Intrusion Detection Sytemsmentioning

confidence: 99%

“…The MRMR feature selection method identified 25 significant features consistent with those identified by Chiba et al [29], as discussed above. The approach adopted by Pallaprolu et al [32] achieved an accuracy of 98% for detecting zero-day intrusion attacks. Abri et al [8] evaluated various types of ML and DL classifiers to detect zero-day attacks and used the Meraz'18 data set to evaluate their models.…”

Section: A Network Intrusion Detection Sytemsmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Zero-Day Intrusion Attacks Using Semi-Supervised Machine Learning Approaches

Mbona

Eloff

2022

IEEE Access

View full text Add to dashboard Cite

Recently, network intrusion attacks, particularly new unknown attacks referred to as zeroday attacks, have become a global phenomenon. Zero-day network intrusion attacks constitute a frequent cybersecurity threat, as they seek to exploit the vulnerabilities of a network system. Previous studies have demonstrated that zero-day attacks can compromise a network for prolonged periods if network traffic analysis (NTA) is not performed thoroughly and efficiently. NTA plays a crucial role in supporting machine learning (ML) based network intrusion detection systems (NIDS) by monitoring and extracting meaningful information from network traffic data. Network traffic data constitute large volumes of data described by features such as destination-to-source packet count. It is important to use only those features that have a significant impact on the performance of an NIDS. The problem is that most existing ML models for NIDS employ features such as Internet protocol (IP) addresses that are redundant for detecting zero-day attacks and therefore negatively impact the performance of these ML models. The solution proposed in this study demonstrates that the law of anomalous numbers, famously known as Benford's law, is a viable technique that can effectively identify significant network features that are indicative of anomalous behaviour and can be used for detecting zero-day attacks. Finally, our study illustrates that semi-supervised ML approaches are effective for detecting zero-day attacks if significant features are optimally chosen. The experimental results demonstrate that one-class support vector machines achieved the best results (Matthews correlation coefficient of 74% and F 1 score of 85%) for detecting zero-day network attacks.

show abstract

Section: A Network Intrusion Detection Sytemsmentioning

confidence: 99%

Section: A Network Intrusion Detection Sytemsmentioning

confidence: 99%

Detecting Zero-Day Intrusion Attacks Using Semi-Supervised Machine Learning Approaches

Mbona

Eloff

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Pallaprolu et al [11] applied Apache Spark Streaming to detect zero-day attacks. Here, the proposed system is tested with the KNN algorithm that showed a precision of 99.57% with a True Positive Rate (TPR) of 94% and a False Positive Rate (FPR) of 3%.…”

Section: Literature Reviewmentioning

confidence: 99%

Comparative Study between Big Data Analysis Techniques in Intrusion Detection

Hafsa

Farah

2018

BDCC

View full text Add to dashboard Cite

Cybersecurity ventures expect that cyber-attack damage costs will rise to $11.5 billion in 2019 and that a business will fall victim to a cyber-attack every 14 seconds. Notice here that the time frame for such an event is seconds. With petabytes of data generated each day, this is a challenging task for traditional intrusion detection systems (IDSs). Protecting sensitive information is a major concern for both businesses and governments. Therefore, the need for a real-time, large-scale and effective IDS is a must. In this work, we present a cloud-based, fault tolerant, scalable and distributed IDS that uses Apache Spark Structured Streaming and its Machine Learning library (MLlib) to detect intrusions in real-time. To demonstrate the efficacy and effectivity of this system, we implement the proposed system within Microsoft Azure Cloud, as it provides both processing power and storage capabilities. A decision tree algorithm is used to predict the nature of incoming data. For this task, the use of the MAWILab dataset as a data source will give better insights about the system capabilities against cyber-attacks. The experimental results showed a 99.95% accuracy and more than 55,175 events per second were processed by the proposed system on a small cluster.

show abstract

“…We test the approach on brand new data, which may exhibit a different behavior from training ( zero day ). This may be typical of detection of streaming data, in which the model has been trained over a past horizon and an abrupt change of the kind of anomalies takes place.…”

Section: Performance Evaluationmentioning

confidence: 99%

Unsupervised learning and rule extraction for Domain Name Server tunneling detection

Aiello

Mongelli

Muselli

et al. 2018

Internet Technology Letters

View full text Add to dashboard Cite

The paper deals with k-means clustering and logic learning machine (LLM) for the detection of Domain Name Server (DNS) tunneling. As the LLM shows more versatility in rule generation and classification precision with respect to traditional decision trees, the approach reveals to be robust to a large set of system conditions. The detection algorithm is designed to be applied over streaming data, without accurate tuning of algorithms' parameters. An extensive performance evaluation is provided with respect to different tunneling tools and applications; silent intruders are considered. Results show robustness on a test set that exhibits a different behavior from training. KEYWORDS covert channel, rule extraction, unsupervised learning 1 Internet Technology Letters. 2019;2:e85.wileyonlinelibrary.com/journal/itl2

show abstract

Zero-Day Attack Identification in Streaming Data Using Semantics and Spark

Cited by 10 publications

References 20 publications

Detecting Zero-Day Intrusion Attacks Using Semi-Supervised Machine Learning Approaches

Detecting Zero-Day Intrusion Attacks Using Semi-Supervised Machine Learning Approaches

Comparative Study between Big Data Analysis Techniques in Intrusion Detection

Unsupervised learning and rule extraction for Domain Name Server tunneling detection

Contact Info

Product

Resources

About