A Spectral Framework for Anomalous Subgraph Detection

Abstract

A wide variety of application domains are concerned with data consisting of entities and their relationships or connections, formally represented as graphs. Within these diverse application areas, a common problem of interest is the detection of a subset of entities whose connectivity is anomalous with respect to the rest of the data. While the detection of such anomalous subgraphs has received a substantial amount of attention, no application-agnostic framework exists for analysis of signal detectability in graph-based data. In this paper, we describe a framework that enables such analysis using the principal eigenspace of a graph's residuals matrix, commonly called the modularity matrix in community detection. Leveraging this analytical tool, we show that the framework has a natural power metric in the spectral norm of the anomalous subgraph's adjacency matrix (signal power) and of the background graph's residuals matrix (noise power). We propose several algorithms based on spectral properties of the residuals matrix, with more computationally expensive techniques providing greater detection power. Detection and identification performance are presented for a number of signal and noise models, including clusters and bipartite foregrounds embedded into simple random backgrounds as well as graphs with community structure and realistic degree distributions. The trends observed verify intuition gleaned from other signal processing areas, such as greater detection power when the signal is embedded within a less active portion of the background. We demonstrate the utility of the proposed techniques in detecting small, highly anomalous subgraphs in real graphs derived from Internet traffic and product co-purchases.Comment: In submission to the IEEE, 16 pages, 8 figure

Statistical evaluation of spectral methods for anomaly detection in static networks

“…The critical factors to consider in an anomaly detection problem are the order of the network, the size of the anomalous subgraph to be detected, and the types of anomalies that are of interest (Miller et al, 2015, Dahan et al, 2017. For example, a small anomalous subgraph is harder to detect than a large anomalous subgraph in the same network (Miller et al, 2015). Also, the type of anomalous subgraphs to detect will significantly affect the efficacy of the proposed method (Miller et al, 2015).…”

“…For example, a small anomalous subgraph is harder to detect than a large anomalous subgraph in the same network (Miller et al, 2015). Also, the type of anomalous subgraphs to detect will significantly affect the efficacy of the proposed method (Miller et al, 2015). Anomaly detection techniques that are robust to these critical factors are, therefore, highly sought after by practitioners.…”

“…In the interest of space, we restrict our analysis to the chi-square and L 1 algorithms. The Sparse PCA method is not covered due to its implementational issues as reported by Miller et al (2015).…”

“…In Miller et al (2015), the only case that is explored for both the chi-square and L 1 algorithms is a binary network with 4096 vertices and average degree of twelve. However, it is possible that the algorithms perform differently when the network order or sparsity of the network changes.…”

Residuals-based subgraph detection with cue vertices

“…Given a large set of entities and their relationships, connections, or interactions, it can be difficult to determine if there is a particular subset of entities that requires special attention [1], [2]. Typically, the objective is to find a relatively small set of vertices whose topology is inconsistent with some notion of expected behavior in the graph.…”