Mutual information applied to anomaly detection

Kopylova, Yuliya; Buell, Duncan A.; Huang, Chin‐Tser; Janies, Jeff

doi:10.1109/jcn.2008.6388332

Cited by 15 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although Tsallis entropy seems to be more popular than Renyi entropy in the context of network anomaly detection the latter was also successfully applied in detection of different anomalies. An example is the work by Yang et al [10] who employed Renyi entropy to early detection of low-rate DDoS attacks and Kopylova et al [11] who reported positive results of using Renyi conditional entropy in detection of selected worms. We believe that with parameterized entropy some limitations of Shannon entropy caused by small descriptive capability [9] which results in a little ability to detect typical small or low-rate anomalies can be overcome.…”

Section: Detection Via Feature Distributionsmentioning

confidence: 99%

“…Therefore, a proper network anomaly detection as one of possible solutions to complement signature-based solutions is essential. Recently, entropy-based methods which rely on network feature distributions has been of a great interest [6][7][8][9][10][11]. It is crucial to check if entropy-based approach is efficient in detection of anomalous network activities caused by modern botnet-like malware [12].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Entropy-Based Network Anomaly Detection Method

Bereziński

Jasiul

Szpyrka

2015

Entropy

163

View full text Add to dashboard Cite

Data mining is an interdisciplinary subfield of computer science involving methods at the intersection of artificial intelligence, machine learning and statistics. One of the data mining tasks is anomaly detection which is the analysis of large quantities of data to identify items, events or observations which do not conform to an expected pattern. Anomaly detection is applicable in a variety of domains, e.g., fraud detection, fault detection, system health monitoring but this article focuses on application of anomaly detection in the field of network intrusion detection.The main goal of the article is to prove that an entropy-based approach is suitable to detect modern botnet-like malware based on anomalous patterns in network. This aim is achieved by realization of the following points: (i) preparation of a concept of original entropy-based network anomaly detection method, (ii) implementation of the method, (iii) preparation of original dataset, (iv) evaluation of the method.

show abstract

Section: Detection Via Feature Distributionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Entropy-Based Network Anomaly Detection Method

Bereziński

Jasiul

Szpyrka

2015

Entropy

163

View full text Add to dashboard Cite

show abstract

“…In opposite, Tellenbach in [21] reported no correlation among header-based features. Parameterized entropy-based approach for network anomaly detection is promising, what is confirmed by Tellenbach [21], who employed Tsallis entropy in his Traffic Entropy Telescope prototype capable to detect a broad spectrum of anomalies, Yang [23], who applied Renyi entropy to early detection of low-rate DDoS attacks detection, and Kopylova [15], who reported positive results of using Renyi conditional entropy in detection of selected fast spreading or aggressive worms. There are some limitations of entropy based detection especially when it comes to detecting small or slow attacks.…”

Section: Related Workmentioning

confidence: 94%

Network Anomaly Detection Using Parameterized Entropy

Bereziński

Szpyrka

Jasiul

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Entropy-based anomaly detection has recently been extensively studied in order to overcome weaknesses of traditional volume and rule based approaches to network flows analysis. From many entropy measures only Shannon, Titchener and parameterized Renyi and Tsallis entropies have been applied to network anomaly detection. In the paper, our method based on parameterized entropy and supervised learning is presented. With this method we are able to detect a broad spectrum of anomalies with low false positive rate. In addition, we provide information revealing the anomaly type. The experimental results suggest that our method performs better than Shannon-based and volume-based approach.Keywords: anomaly detection, entropy, netflow, network traffic measurement IntroductionThe number of anomalies in IP networks caused by wormlike activities is growing [2]. Widely used security solutions based on signatures or rules like firewalls, antiviruses and intrusion detection systems do not provide sufficient protection because they do not cope with evasion techniques and not known yet (0-day) attacks [12], [13]. Therefore, network anomaly detection as one of possible solutions is becoming an essential area of research. Anomaly detection is an identification of observations which do not conform to an expected behavior. In a supervised anomaly detection a labeled data set that involves training a classifier is required.There are many problems with anomaly detectors which have to be addressed. The main challenge is setting up a precise boundary between normal and anomalous behavior to avoid high false positive error rate or low detection rate. Another problems are long computation time, anomaly details extraction and root-cause identification [7]. In our previous work [4], some generalizations of entropy were described in details and preliminary results of using parameterized entropies were presented. In this paper, we make two major contributions. Firstly, we present our method and results in comparison with Shannon-based and volume-based approach. Secondly, we describe data set as well as the method we used to generate anomalies.

show abstract

“…In terms of research, we need to focus more on multivariate interactions; e.g., removing individual variable effects using mutual information. [43,44] Finally, the applied metrics need to be characterized with respect to alternative statistical methods such as effect size analysis to facilitate practical interpretation of the metrics.…”

Section: Future Workmentioning

confidence: 99%