MemStream: Memory-Based Streaming Anomaly Detection

Bhatia, Siddharth; Jain, Arjit; Srivastava, Shivin; Kawaguchi, Kenji; Hooi, Bryan

doi:10.1145/3485447.3512221

Cited by 14 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…USCDD-AE and DEVDAN are mainly based on the active concept drift adaptation method but ignore mutation oblivion when adding new layers. MemStream [61] is used for anomaly detection in multidimensional data and concept drift. It first uses a small portion of the training set and extracts features using the denoising autoencoder.…”

Section: Concept Drift Adaptation Methods Based On Generative Learningmentioning

confidence: 99%

Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review

Xiang

Cong

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

With the advent of the fourth industrial revolution, data-driven decision making has also become an integral part of decision making. At the same time, deep learning is one of the core technologies of the fourth industrial revolution that have become vital in decision making. However, in the era of epidemics and big data, the volume of data has increased dramatically while the sources have become progressively more complex, making data distribution highly susceptible to change. These situations can easily lead to concept drift, which directly affects the effectiveness of prediction models. How to cope with such complex situations and make timely and accurate decisions from multiple perspectives is a challenging research issue. To address this challenge, we summarize concept drift adaptation methods under the deep learning framework, which is beneficial to help decision makers make better decisions and analyze the causes of concept drift. First, we provide an overall introduction to concept drift, including the definition, causes, types, and process of concept drift adaptation methods under the deep learning framework. Second, we summarize concept drift adaptation methods in terms of discriminative learning, generative learning, hybrid learning, and others. For each aspect, we elaborate on the update modes, detection modes, and adaptation drift types of concept drift adaptation methods. In addition, we briefly describe the characteristics and application fields of deep learning algorithms using concept drift adaptation methods. Finally, we summarize common datasets and evaluation metrics and present future directions.

show abstract

Section: Concept Drift Adaptation Methods Based On Generative Learningmentioning

confidence: 99%

Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review

Xiang

Cong

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…[20,45,62,79,80] use a stream of multi-aspect records as input. MemStream [21] can learn dynamically changing trends to handle time-varying data distribution known as concept drift [27,35,57]. Although these methods have the ability to detect multiple anomalies, they cannot identify the types of anomalies or capture dynamical multi-aspect patterns.…”

Section: Related Workmentioning

confidence: 99%

“…where |S −1 𝑟 | is the total segment length of the regime 𝜃 𝑟 . Note that, in data streams, the concept of normal changes over time, and this is known as concept drift [21]. This approach can adaptively change the norm to judge incoming tensors as the concept drift.…”

Section: C-compressormentioning

confidence: 99%

“…For NTM, we implemented it based on the pytorch framework and applied Adam optimization with a learning rate of 1𝑒 − 3, following the design and the parameter setting in [85]. We also used open-sourced implementations of TICC [37], T-LSTM [17], RRCF [34], CubeMarker [39], and MemStream [21], provided by the authors, following parameter settings as suggested in the original papers. For a fair comparison in terms of computational time, we implemented TriMine in Python, following C implementation provided by the authors.…”

Section: Symbolmentioning

confidence: 99%

See 1 more Smart Citation

Fast and Multi-aspect Mining of Complex Time-stamped Event Streams

Nakamura

Matsubara

Kawabata

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

Given a huge, online stream of time-evolving events with multiple attributes, such as online shopping logs: (item, price, brand, time), how can we summarize large, dynamic high-order tensor streams? How can we see any hidden patterns, rules, and anomalies? Our answer is to focus on two types of patterns, i.e., "regimes" and "components", over high-order tensor streams, for which we present an efficient and effective method, namely CubeScope. Specifically, it identifies any sudden discontinuity and recognizes distinct dynamical patterns, "regimes" (e.g., weekday/weekend/holiday patterns).In each regime, it also performs multi-way summarization for all attributes (e.g., item, price, brand, and time) and discovers hidden "components" representing latent groups (e.g., item/brand groups) and their relationship. Thanks to its concise but effective summarization, CubeScope can also detect the sudden appearance of anomalies and identify the types of anomalies that occur in practice.Our proposed method has the following properties: (a) Effective: it captures dynamical multi-aspect patterns, i.e., regimes and components, and statistically summarizes all the events; (b) General: it is practical for successful application to data compression, pattern discovery, and anomaly detection on various types of tensor streams; (c) Scalable: our algorithm does not depend on the length of the data stream and its dimensionality. Extensive experiments on real datasets demonstrate that CubeScope finds meaningful patterns and anomalies correctly, and consistently outperforms the state-of-the-art methods as regards accuracy and execution speed.

show abstract