Toward a more practical unsupervised anomaly detection system

Song, Jungsuk; Takakura, Hiroki; Okabe, Yasuo; Nakao, Koji

doi:10.1016/j.ins.2011.08.011

Cited by 84 publications

(39 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…So, there may be a difficulty for a network manager to tune and optimize the required parameters based on changing behavior of network characteristics. Jungsuk Song et al [12], presented a more practical unsupervised IDS and evaluated with real traffic data collected from Kytoto University honeypots.…”

Section: Related Workmentioning

confidence: 99%

Partial Least Square based Improved Intrusion Detection System

Sangve¹,

Kulkarni²

2017

IJCA

View full text Add to dashboard Cite

Various Artificial Intelligence (AI) based computing techniques for intrusion detection has been proposed using popular large-scale datasets like DARPA 98 and KDD Cup 99. However, AI based systems such as using representative instances are computationally inefficient. In this paper, the computationally efficient approach is proposed for anomaly detection by combining Partial Least Square (PLS) and technique of extracting representative instances. The PLS helps in feature selection and provides dimensionality reduction. Further, to decline the processing time the representative instances are properly chosen from the data set before classification. The classic instances are selected from the subsets of data which are obtained by Centroid-based partitioning technique. The system utilizes these paradigmatic instances as a training set. Finally, KNN classifier is trained using these paradigmatic instances. The results obtained using the proposed approach indicates a considerable fall in the processing time and space utilization.

show abstract

Section: Related Workmentioning

confidence: 99%

Partial Least Square based Improved Intrusion Detection System

Sangve¹,

Kulkarni²

2017

IJCA

View full text Add to dashboard Cite

show abstract

“…The clustering problems were handled by calculating the initialization parameters based on log events clustered and the suitability of the parameters was tested with different settings to evaluate the accuracy in producing better clusters. Unlike [7,11], clustering was used twice at different points in UHAD to classify the events: 1) to assist the filterer to identify the abnormal events and 2) to support the analysis method in detecting anomalies.…”

Section: Related Workmentioning

confidence: 99%

An unsupervised heterogeneous log-based framework for anomaly detection

Hajamydeen¹,

Udzir²,

Mahmod³

et al. 2016

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Log analysis is a method to identify intrusions at the host or network level by scrutinizing the log events recorded by the operating systems, applications, and devices. Most work contemplates a single type of log for analysis, leading to an unclear picture of the situation and difficulty in deciding the existence of an intrusion. Moreover, most existing detection methods are knowledge-dependent, i.e. using either the characteristics of an anomaly or the baseline of normal traffic behavior, which limits the detection process to only anomalies based on the acquired knowledge. To discover a wide range of anomalies by scrutinizing various logs, this paper presents a new unsupervised framework, UHAD, which uses a two-step strategy to cluster the log events and then uses a filtering threshold to reduce the volume of events for analysis. The events from heterogeneous logs are assembled together into a common format and are analyzed based on their features to identify anomalies. Clustering accuracy of K-means, expectation-maximization, and farthest first were compared and the impact of clustering was captured in all the subsequent phases. Even though log events pass through several phases in UHAD before being concluded as anomalous, experiments have shown that the selection of the clustering algorithm and the filtering threshold significantly influences the decision. The framework detected the majority of anomalies by relating the events from heterogeneous logs. Specifically, the usage of K-means and expectationmaximization supported the framework to detect an average of 87.26% and 85.24% anomalous events respectively with various subsets.

show abstract

“…It is able to construct intrusion detection models automatically without using labeled training data. In [22] the authors have optimized the values of parameters without predefining. This helps to construct models based on without tuning the parameters, and thus contributes to more practical operations in the real environment.…”

Section: Related Workmentioning

confidence: 99%

A Hybrid Real-time Zero-day Attack Detection and Analysis System

Kaur

Singh²

2015

IJCNIS

View full text Add to dashboard Cite

Abstract-A zero-day attack poses a serious threat to the Internet security as it exploits zero-day vulnerabilities in the computer systems. Attackers take advantage of the unknown nature of zero-day exploits and use them in conjunction with highly sophisticated and targeted attacks to achieve stealthiness with respect to standard intrusion detection techniques. Thus, it's difficult to defend against such attacks. Present research exhibits various issues and is not able to provide complete solution for the detection and analysis of zero-day attacks. This paper presents a novel hybrid system that integrates anomaly, behavior and signature based techniques for detecting and analyzing zero-day attacks in real-time. It has layered and modular design which helps to achieve high performance, flexibility and scalability. The system is implemented and evaluated against various standard metrics like True Positive Rate (TPR), False Positive Rate (FPR), FMeasure, Total Accuracy (ACC) and Receiver Operating Characteristic (ROC) curve. The result shows high detection rate with nearly zero false positives. Additionally, the proposed system is compared with Honeynet system.

show abstract

Toward a more practical unsupervised anomaly detection system

Cited by 84 publications

References 12 publications

Partial Least Square based Improved Intrusion Detection System

Partial Least Square based Improved Intrusion Detection System

An unsupervised heterogeneous log-based framework for anomaly detection

A Hybrid Real-time Zero-day Attack Detection and Analysis System

Contact Info

Product

Resources

About