Parametric Methods for Anomaly Detection in Aggregate Traffic

Thatte, Gautam; Mitra, Urbashi; Heidemann, John

doi:10.1109/tnet.2010.2070845

Cited by 121 publications

(82 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For anomaly detection, a statistical model is fitted to normal data, and a statistical inference test detects any abnormal behavior. Thatte et al use packet-size statistics and traffic rates to build a statistical model, and then employ the sequential probability ratio test (SPRT) in the detection phase [Thatte et al, 2011].…”

Section: Methodsmentioning

confidence: 99%

Incremental anomaly detection using two-layer cluster-based structure

Bigdeli

Mohammadi

Raahemi

et al. 2018

Information Sciences

View full text Add to dashboard Cite

Anomaly detection algorithms face several challenges, including processing speed and dealing with noise in data. In this thesis, a two-layer clusterbased anomaly detection structure is presented which is fast, noise-resilient and incremental. In this structure, each normal pattern is considered as a cluster, and each cluster is represented using a Gaussian Mixture Model (GMM). Then, new instances are presented to the GMM to be labeled as normal or abnormal.The proposed structure comprises three main steps. In the first step, the data are clustered. The second step is to represent each cluster in a way that enables the model to classify new instances. The Summarization based on Gaussian Mixture Model (SGMM) proposed in this thesis represents each cluster as a GMM.In the third step, a two-layer structure efficiently updates clusters using In most real-time anomaly detection applications, incoming instances are often similar to previous ones. In these cases, there is no need to update clusters based on duplicates, since they have already been modeled in the cluster distribution. The two-layer structure is responsible for identifying redundant instances. In this structure, redundant instance are ignored, and the remaining new instances are used to update clusters. Ignoring redundant instances, which are typically in the majority, makes the detection phase fast.Each part of the general structure is validated in this thesis. The experiments include, detection rates, clustering goodness, time, memory usage and the complexity of the algorithms. The accuracy of the clustering and summarization of clusters using GMMs is evaluated, and compared to that of other methods. Using Davies-Bouldin (DB) and Dunn indexes, the distances for original and regenerated clusters using GMMs is almost zero with SGMM method while this value for ABACUS is around 0.01. Moreover, the results show that the SGMM algorithm is 3 times faster than ABACUS in running time, using one-third of the memory used by ABACUS.The CPL method, used to label new instances, is found to collectively remove the effect of noise, while increasing the accuracy of labeling new instances. In a noisy environment, the detection rate of the CPL method is 5% higher than other algorithms such as one-class SVM. The false alarm iii rate is decreased by 10% on average. Memory use is 20 times lesser that that of the one-class SVM.The proposed method is found to lower the false alarm rate, which is one of the basic problems for the one-class SVM. Experiments show the false alarm rate is decreased from 5% to 15% among different datasets, while the detection rate is increased from 5% to 10% in different datasets with twolayer structure. The memory usage for the two-layer structure is 20 to 50 times less than that of one-class SVM. One-class SVM uses support vectors in labeling new instances, while the labeling of the two-layer structure depends on the number of GMMs. The experiments show that the two-layer structure is 20 to 50 times faster than the one-class SVM in labelin...

show abstract

Section: Methodsmentioning

confidence: 99%

Incremental anomaly detection using two-layer cluster-based structure

Bigdeli

Mohammadi

Raahemi

et al. 2018

Information Sciences

View full text Add to dashboard Cite

show abstract

“…Changes in network's aggregate traffic anomalies [79] Observing packet size and traffic rate parameters through proposed bPDM mechanism to calculate probability ratio test.…”

Section: Basis Of Defense Methodsmentioning

confidence: 99%

“…In [79], authors devise a mechanism of parametric methods to detect anomalies in network traffic using aggregate traffic properties without any need of flow separation. The mechanism developed is called bivariate Parametric Detection Mechanism (bPDM).…”

Section: Defense Against Application Layer Ddos Attacksmentioning

confidence: 99%

A Survey on DDoS Attack and Defense Strategies: From Traditional Schemes to Current Techniques

Aamir

Zaidi

2013

IIS

View full text Add to dashboard Cite

Distributed Denial of Service (DDoS) attacks exhaust victim's bandwidth or services. Traditional architecture of Internet is vulnerable to DDoS attacks and an ongoing cycle of attack & defense is observed. A recent attack report of year 2013 -'Quarter 1' from Prolexic Technologies identifies that 1.75 percent increase in total number of DDoS attacks has been recorded as compared to similar attacks of previous year's last quarter. In this paper, different types and techniques of DDoS attacks and their countermeasures are surveyed. The significance of this paper is the coverage of many aspects of countering DDoS attacks including new research on the topic. We survey different papers describing methods of defense against DDoS attacks based on entropy variations, traffic anomaly parameters, neural networks, device level defense, botnet flux identifications, application layer DDoS defense and countermeasures in wireless networks, CCN & cloud computing environments. We also discuss some traditional methods of defense such as traceback and packet filtering techniques, so that readers can identify major differences between traditional and current techniques of defense against DDoS attacks. We identify that application layer DDoS attacks possess the ability to produce greater impact on the victim as they are driven by legitimate-like traffic, making it quite difficult to identify and distinguish from legitimate requests. The need of improved defense against such attacks is therefore more demanding in research. The study conducted in this paper can be helpful for readers and researchers to recognize better techniques of defense in current times against DDoS attacks and contribute with more research on this topic in the light of future challenges identified in this paper.

show abstract

“…This statistical-based detection technique provides a solution to detect outgoing anomalous traffic at source networks. Thatte et al [7] developed a Bivariate Parametric Detection Mechanism (BPDM) operating on aggregate traffic. The BPDM is engaged in the Sequential Probability Ratio Test (SPRT) on two aggregate traffic statistics (i.e., packet size and packet rate), and it maintain an anomaly only when a rise in the traffic volume is associated with a change in the distribution of packet-size.…”

Section: Anomaly-based Detectionmentioning

confidence: 99%