TellTail: Fast Scoring and Detection of Dense Subgraphs

Abstract

Suppose you visit an e-commerce site, and see that 50 users each reviewed almost all of the same 500 products several times each: would you get suspicious? Similarly, given a Twitter follow graph, how can we design principled measures for identifying surprisingly dense subgraphs? Dense subgraphs often indicate interesting structure, such as network attacks in network traffic graphs. However, most existing dense subgraph measures either do not model normal variation, or model it using an Erdős-Renyi assumption - but this assumption has been discredited decades ago. What is the right assumption then? We propose a novel application of extreme value theory to the dense subgraph problem, which allows us to propose measures and algorithms which evaluate the surprisingness of a subgraph probabilistically, without requiring restrictive assumptions (e.g. Erdős-Renyi). We then improve the practicality of our approach by incorporating empirical observations about dense subgraph patterns in real graphs, and by proposing a fast pruning-based search algorithm. Our approach (a) provides theoretical guarantees of consistency, (b) scales quasi-linearly, and (c) outperforms baselines in synthetic and ground truth settings.

Weakening the Inner Strength: Spotting Core Collusive Users in YouTube Blackmarket Network

“…Shin, Eliassi-Rad, and Faloutsos (2016) showed empirical patterns in real-world graphs related to k-cores. Recently, it has been observed that the subgraphs of the detected core users are used for several graph-related tasks, such as community detection (Peng, Kolda, and Pinar 2014), dense-subgraph detection (Hooi et al 2020) etc. We encourage the readers to go through Malliaros, Papadopoulos, and Vazirgiannis (2016) for a comprehensive survey on network core detection.…”

CubeFlow: Money Laundering Detection with Coupled Tensors

“…Note that although CPD works well at some densities, it fluctuate greatly, indicating that CPD is not very suitable for ML behavior detection. Flow Surprisingness estimation with extreme value theory: Inspired by [4], we use Generalized Pareto (GP) Distribution, a commonly used probability distribution within extreme value theory, to estimate the extreme tail of a distribution without making strong assumptions about the distribution it- self. GP distributions exhibit heavy-tailed decay (i.e.…”