Applying PCA for Traffic Anomaly Detection: Problems and Solutions

Abstract

Abstract-Spatial Principal Component Analysis (PCA) has been proposed for network-wide anomaly detection. A recent work has shown that PCA is very sensitive to calibration settings, unfortunately, the authors did not provide further explanations for this observation. In this paper, we fill this gap and provide the reasoning behind the found discrepancies.First, we revisit PCA for anomaly detection and evaluate its performance on our data. We develop a slightly modified version of PCA that uses only data from a single router. Instead of correlating data across different spatial measurement points, we correlate the data across different metrics. With the help of the analyzed data, we explain the pitfalls of PCA and underline our argumentation with measurement results. We show that the main problems that make PCA difficult to apply are (i) the temporal correlation in the data; (ii) the non-stationarity of the data; and (iii) the difficulty about choosing the right number of components. Moreover, we propose a solution to deal with the most dominant problem, the temporal correlation in data. We find that when we consider temporal correlation, PCA detection results are significantly improved.

An Entropy-Based Network Anomaly Detection Method

“…According to Brauckhoff et al [123], a realistic simulation of legitimate traffic is largely an unsolved problem today and one of the solution is combining generated anomalies with real, legitimate traffic traces. In [123] and then in [124], Brauckhoff et al introduced the FLAME tool for injection of hand-crafted anomalies into a given background traffic trace. This tool is freely available but the current distribution does not include any models reflecting anomalies.…”

Discovering spatio-temporal causal interactions in traffic data streams

“…However, there have been a number of efforts on detecting only outliers from spatially and temporally distributed data. For example, principle component analysis (PCA) has been used for network-wide anomaly detection [13,26,20,1]. However, PCA results (as we will show) cannot capture volume heterogeneity and are also very sensitive to parameter settings which are highly data dependent.…”

Anomaly Detection of Network Traffic Based on Prediction and Self-Adaptive Threshold

“…Then, Asrul [12] introduced ARMIA model for predicting traffic and detect anomaly. Brauckhoff [13] employed PCA analysis for traffic anomaly detection.…”