Combining sketch and wavelet models for anomaly detection

Abstract

Statistics-based anomaly detections have been studied and implemented widely due to their potential to discover unseen anomalies. Several data analyzing techniques such as Wavelet have been successfully applied to this field of research. Wavelet analysis is one of the popular techniques that can be used to extract unusual patterns hidden within time-series data. When combined with the powerful data summarization technique like sketch, it could be able to detection significant changes in network data without any prior knowledge about the targeted traffic. In this paper, we study the anomaly detection approach based on the combination of random projection (sketch) and wavelet analysis. We apply our proposed algorithm to the traffic traces collected on the trans-Pacific transit backbone link (MAWI dataset), and compare it with other algorithms [1] [2], and the port heuristic methodology. The experimental results show that our algorithm can detect and identify a large number of anomalous traffic that are, for example, associated with some malware activities. Moreover, our proposed algorithm is also capable of detecting low-intensity anomalies as well as some types of malicious traffic that cannot be identified by the traditional wavelet analysis.

Performance evaluation of sketch schemes on traffic anomaly detection accuracy

“…Previous studies concerning sketch schemes did not clarify the impact of sketch parameters on the anomaly detection performance [2,3]. Thus, our research aimed to evaluate the processing time and traffic anomaly detection accuracy using a sketch as a function of two sketch parameters: the number of hash functions and hash table size.…”

Study of Self-Similarity for Detection of Rate-based Network Anomalies

“…The observed sketch is compared with the forecasted model and result is declared accordingly. Some of the pioneer works in this area have been done by S. Pukkawanna & K. Fukuda in [13]. They have used sketches to identify source IP addresses of anomalous traffic efficiently.…”

Detecting Anomalies in Massive Traffic Streams Based on S-Transform Analysis of Summarized Traffic Entropies

“…Then, they filtered only a set of scales and detected anomalies in the Wavelet coefficients. The works in [9], [12], [24] enhanced their DWT-based anomaly detectors by adding traffic sketching before performing anomaly detection. The authors in [4] applied Continuous Wavelet Transform (CWT) instead of DWT to detect volume-based network anomalies.…”

“…The authors in [8] utilized the sketch technique to randomly aggregate traffic flows and then used CUmulative SUM (CUSUM) to identify change points. The sketch technique was also applied together with PCA [27] or Wavelet transform [9], [12], [24] to detect unusual events.…”