F-FADE: Frequency Factorization for Anomaly Detection in Edge Streams

Chang, Yung Chun; Li, Pan; Sosič, Rok; Afifi, Mansur; Schweighauser, Marco; Leskovec, Jure

doi:10.1145/3437963.3441806

Cited by 36 publications

(23 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PENminer [8] explores the persistence of activity snippets, i.e., the length and regularity of edge-update sequences' reoccurrences. F-FADE [15] aims to detect anomalous interaction patterns by factorizing the frequency of those patterns. These methods can effectively detect periodic patterns, but they require a considerable amount of time to explore and maintain possible patterns in the network.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Isconna: Streaming Anomaly Detection with Frequency and Patterns

Li¹,

Bhatia²,

Hooi³

2021

Preprint

View full text Add to dashboard Cite

An edge stream is a common form of presentation of dynamic networks. It can evolve with time, with new types of nodes or edges being continuously added. Existing methods for anomaly detection rely on edge occurrence counts or compare pattern snippets found in historical records. In this work, we propose Isconna, which focuses on both the frequency and the pattern of edge records. The burst detection component targets anomalies between individual timestamps, while the pattern detection component highlights anomalies across segments of timestamps. These two components together produce three intermediate scores, which are aggregated into the final anomaly score. Isconna does not actively explore or maintain pattern snippets; it instead measures the consecutive presence and absence of edge records. Isconna is an online algorithm, it does not keep the original information of edge records; only statistical values are maintained in a few count-min sketches (CMS). Isconna's space complexity O(rc) is determined by two user-specific parameters, the size of CMSs. In worst case, Isconna's time complexity can be up to O(rc), but it can be amortized in practice. Experiments show that Isconna outperforms five state-of-the-art frequency-and/or pattern-based baselines on six real-world datasets with up to 20 million edge records.

show abstract

Section: Related Workmentioning

confidence: 99%

“…On the other hand, there is another category of algorithms that aims to detect anomalies using patterns or motifs [8,15,43,38,30,33,55,3,23,29,41,40,35,31,7,20,11]. However, many of these methods require active exploration of patterns or snippets, increasing memory and time requirements.…”

Section: Introductionmentioning

confidence: 99%

Isconna: Streaming Anomaly Detection with Frequency and Patterns

Li¹,

Bhatia²,

Hooi³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Though the variants of Isconna demonstrates a comparatively better accuracy than the state-of-the-art, however it suffers from parameter tuning. Very recently, Chang et al have proposed a frequency factorization approach (termed as F-fade) for anomaly detection in edge streams [5].…”

Section: Related Workmentioning

confidence: 99%

Distributed Anomaly Detection in Edge Streams using Frequency based Sketch Datastructures

Chanda¹,

Bhattacharya²

2021

Preprint

View full text Add to dashboard Cite

Often logs hosted in large data centers represent network traffic data over a long period of time. For instance, such network traffic data logged via a TCP dump packet sniffer (as considered in the 1998 DARPA intrusion attack) included network packets being transmitted between computers. While an online framework is necessary for detecting any anomalous or suspicious network activities like denial of service attacks or unauthorized usage in real time, often such large data centers log data over long periods of time (e.g., TCP dump) and hence an offline framework is much more suitable in such scenarios. Given a network log history of edges from a dynamic graph, how can we assign anomaly scores to individual edges indicating suspicious events with high accuracy using only constant memory and within limited time than state-of-the-art methods? We propose MDistrib and its variants which provides (a) faster detection of anomalous events via distributed processing with GPU support compared to other approaches, (b) better false positive guarantees than state of the art methods considering fixed space and (c) with collision aware based anomaly scoring for better accuracy results than state-of-theart approaches. We describe experiments confirming that MDistrib is more efficient than prior work.

show abstract

“…MIDAS [15] proposes a probabilistic framework to detect microcluster anomalies (suddenly arriving of groups of suspiciously similar edges). F-FADE [13] uses a frequency-factorization technique to model the distributions of edge frequencies and flags edges with low likelihood of the observed frequency as anomaly.…”

Section: Related Workmentioning

confidence: 99%

“…More recently, [11], [12], [13] have proposed to detect individually surprising edges in the edge stream which have superior memory and time efficiency. However, due to the non-i.i.d.…”

Section: Introductionmentioning

confidence: 99%

Anomalous Edge Detection in Edge Exchangeable Social Network Models

Luo,

Nettasinghe,

Krishnamurthy

2021

Preprint

View full text Add to dashboard Cite

This paper studies detecting anomalous edges in directed graphs that model social networks. We exploit edge exchangeability as a criterion for distinguishing anomalous edges from normal edges. Then we present an anomaly detector based on conformal prediction theory; this detector has a guaranteed upper bound for false positive rate. In numerical experiments, we show that the proposed algorithm achieves superior performance to baseline methods.

show abstract

F-FADE: Frequency Factorization for Anomaly Detection in Edge Streams

Cited by 36 publications

References 25 publications

Isconna: Streaming Anomaly Detection with Frequency and Patterns

Isconna: Streaming Anomaly Detection with Frequency and Patterns

Distributed Anomaly Detection in Edge Streams using Frequency based Sketch Datastructures

Anomalous Edge Detection in Edge Exchangeable Social Network Models

Contact Info

Product

Resources

About