Intelligent Detection of Large-Scale KPI Streams Anomaly Based on Transfer Learning

Duan, Xiaoyan; Chen, Ningjiang; Xie, Yongsheng

doi:10.1007/978-981-15-1899-7_26

Cited by 4 publications

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Publications Sparse Autoencoder [139], [141]- [143], ... Denoising Autoencoder (DAE) [19], [65], [130], [143], [144], ... Variational Autoencoder (VAE) [145], [146], ... Shared Hidden Layer AE (SHLAE) [19], [145], [147], [148], ... Convolutional Autoencoder [146], [149]- [151], ...…”

Section: Autoencoder Typementioning

confidence: 99%

Transfer Learning With Time Series Data: A Systematic Mapping Study

Weber

Auch

Doblander³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Transfer Learning is a well-studied concept in machine learning, that relaxes the assumption that training and testing data need to be drawn from the same distribution. Recent success in applying transfer learning in the area of computer vision has motivated research on transfer learning also in context of time series data. This benefits learning in various time series domains, including a variety of domains based on sensor values. In this paper, we conduct a systematic mapping study of literature on transfer learning with time series data. Following the review guidelines of Kitchenham and Charters, we identify and analyze 223 relevant publications. We describe the pursued approaches and point out trends. Especially during the last two years, there has been a vast increase in the number of publications on the topic. This paper's findings can help researchers as well as practitioners getting into the field and can help identify research gaps.

show abstract

Section: Autoencoder Typementioning

confidence: 99%

Transfer Learning With Time Series Data: A Systematic Mapping Study

Weber

Auch

Doblander³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

Distributed and explainable GHSOM for anomaly detection in sensor networks

Mignone,

Corizzo,

Ceci

2024

Mach Learn

View full text Add to dashboard Cite

The identification of anomalous activities is a challenging and crucially important task in sensor networks. This task is becoming increasingly complex with the increasing volume of data generated in real-world domains, and greatly benefits from the use of predictive models to identify anomalies in real time. A key use case for this task is the identification of misbehavior that may be caused by involuntary faults or deliberate actions. However, currently adopted anomaly detection methods are often affected by limitations such as the inability to analyze large-scale data, a reduced effectiveness when data presents multiple densities, a strong dependence on user-defined threshold configurations, and a lack of explainability in the extracted predictions. In this paper, we propose a distributed deep learning method that extends growing hierarchical self-organizing maps, originally designed for clustering tasks, to address anomaly detection tasks. The SOM-based modeling capabilities of the method enable the analysis of data with multiple densities, by exploiting multiple SOMs organized as a hierarchy. Our map-reduce implementation under Apache Spark allows the method to process and analyze large-scale sensor network data. An automatic threshold-tuning strategy reduces user efforts and increases the robustness of the method with respect to noisy instances. Moreover, an explainability component resorting to instance-based feature ranking emphasizes the most salient features influencing the decisions of the anomaly detection model, supporting users in their understanding of raised alerts. Experiments are conducted on five real-world sensor network datasets, including wind and photovoltaic energy production, vehicular traffic, and pedestrian flows. Our results show that the proposed method outperforms state-of-the-art anomaly detection competitors. Furthermore, a scalability analysis reveals that the method is able to scale linearly as the data volume presented increases, leveraging multiple worker nodes in a distributed computing setting. Qualitative analyses on the level of anomalous pollen in the air further emphasize the effectiveness of our proposed method, and its potential in determining the level of danger in raised alerts.

show abstract

Efficient KPI Anomaly Detection Through Transfer Learning for Large-Scale Web Services

Zhang

Zhong

et al. 2022

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

Intelligent Detection of Large-Scale KPI Streams Anomaly Based on Transfer Learning

Cited by 4 publications

References 9 publications

Transfer Learning With Time Series Data: A Systematic Mapping Study

Transfer Learning With Time Series Data: A Systematic Mapping Study

Distributed and explainable GHSOM for anomaly detection in sensor networks

Efficient KPI Anomaly Detection Through Transfer Learning for Large-Scale Web Services

Contact Info

Product

Resources

About