A theory and algorithm for detecting and classifying weak, distributed patterns in network data is presented. The patterns we consider are anomalous temporal correlations between signals recorded at sensor nodes in a network. We use robust matrix completion and second order analysis to detect distributed patterns that are not discernible at the level of individual sensors. When viewed independently, the data at each node cannot provide a definitive determination of the underlying pattern, but when fused with data from across the network the relevant patterns emerge. We are specifically interested in detecting weak patterns in computer networks where the nodes (terminals, routers, servers, etc.) are sensors that provide measurements (of packet rates, user activity, central processing unit usage, etc.). The approach is applicable to many other types of sensor networks including wireless networks, mobile sensor networks, and social networks where correlated phenomena are of interest.

A new approach based on graph wavelets for analyzing the spatial and temporal behavior of Internet traffic anomalies is presented. This approach is applied to Internet2 traffic measurements to evaluate the time duration and spatial spread (number of links affected) of anomalies. Based on the empirical results, a node model is proposed that captures the behavior of anomalies at individual network nodes. The model considers various aspects of anomalies, such as its origin, termination, propagation, duration and volume changes. The derivation of the model parameters requires only local node information, but the model is capable of producing network-wide anomalies whose behavior mimics network wide anomalies. Model is verified by using Internet2 traffic data. Since the proposed model can be specified using only a few parameters, it can be used in place of large anomaly traces with a great data reduction. As extensions, the model is applied over a path and an aggregated model that applies to a neighborhood in the network is also presented. A method to use the graph wavelet components found during the analysis to implement a real-time anomaly monitoring system is also discussed. keywords-Graph wavelets; Internet traffic anomalies; Modeling network anomalies

No abstract

Models for Internet traffic anomalies greatly benefit a range of applications including robust network design, network provisioning and performance studies. A novel approach to analyse and model network traffic anomalies is presented. The proposed approach individually characterises different aspects of anomalies, such as origin, termination, propagation and changes in duration and volume, with common random processes. These characteristics are then integrated into a single model that successfully captures the overall anomaly behaviours. Characterisation of each anomaly property requires only a few parameters, leading to a concise set of parameters for the entire model. Although the model is calibrated with local measurements made at nodes, it successfully represents the global behaviours of anomalies over the network. The proposed model is applicable both at nodal level and at subnet level. This enables hierarchically analysing large and sophisticated networks. Anomalies are analysed using a multi-scale analysis framework based on which, a real-time monitoring system that efficiently communicate ongoing anomaly information across the network is developed. The system is also used for learning regional model parameters distributively. Internet2 traffic data is analysed using the framework, and the corresponding model parameters are derived. These results provide insight on the nature of anomalies in networks.

No abstract