Towards a Rigorous Evaluation of Time-series Anomaly Detection

Kim, Siwon; Choi, Kukjin; Choi, Hyun-Soo; Lee, Byunghan; Yoon, Sungroh

doi:10.48550/arxiv.2109.05257

Cited by 3 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intrinsic Anomaly Detection Performance Acknowledging recent concern on the validity of time series anomaly benchmarks (Kim et al, 2021), our main quantitative results report AUROC scores on the intrinsic anomaly detection task. For the ResThresh method, we compute the AUROC score based on the maximum residual value over the full residual series.…”

Section: Quantitative Resultsmentioning

confidence: 99%

Intrinsic Anomaly Detection for Multi-Variate Time Series

Rabanser¹,

Januschowski²,

Rasul³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a novel, practically relevant variation of the anomaly detection problem in multi-variate time series: intrinsic anomaly detection. It appears in diverse practical scenarios ranging from DevOps to IoT, where we want to recognize failures of a system that operates under the influence of a surrounding environment. Intrinsic anomalies are changes in the functional dependency structure between time series that represent an environment and time series that represent the internal state of a system that is placed in said environment. We formalize this problem, provide under-studied public and new purpose-built data sets for it, and present methods that handle intrinsic anomaly detection. These address the shortcoming of existing anomaly detection methods that cannot differentiate between expected changes in the system's state and unexpected ones, i.e., changes in the system that deviate from the environment's influence. Our most promising approach is fully unsupervised and combines adversarial learning and time series representation learning, thereby addressing problems such as label sparsity and subjectivity, while allowing to navigate and improve notoriously problematic anomaly detection data sets.

show abstract

Section: Quantitative Resultsmentioning

confidence: 99%

Intrinsic Anomaly Detection for Multi-Variate Time Series

Rabanser¹,

Januschowski²,

Rasul³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For each method, we perform a search for the best F 1 -score by considering all unique scores a method generated for the data set at hand as decision thresholds. We are well aware of the valid criticism of this evaluation method by Kim et al [30], however we propose here to demonstrate that our method performs reasonably rather than set a new state-of-the-art result. We think that for our purposes this metric is sufficient.…”

Section: Detection Performancementioning

confidence: 88%

Online Time Series Anomaly Detection with State Space Gaussian Processes

Bock¹,

Aubet²,

Gasthaus³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose r-ssGPFA, an unsupervised online anomaly detection model for uniand multivariate time series building on the efficient state space formulation of Gaussian processes. For high-dimensional time series, we propose an extension of Gaussian process factor analysis to identify the common latent processes of the time series, allowing us to detect anomalies efficiently in an interpretable manner. We gain explainability while speeding up computations by imposing an orthogonality constraint on the mapping from the latent to the observed. Our model's robustness is improved by using a simple heuristic to skip Kalman updates when encountering anomalous observations. We investigate the behaviour of our model on synthetic data and show on standard benchmark datasets that our method is competitive with state-of-the-art methods while being computationally cheaper.

show abstract

“…We chose not to evaluate this setting as it has been shown that current benchmarks can be dominated by trivial methods. Particularly, Kim et al (2021) show that a simple approach of using the L 2 norm of the raw features is sufficient for achieving competitive or better than the state-of-the-art on popular datasets such as WADI or SMAP. Furthermore, very strong results using the point-adjust protocol can be achieved using a random baseline.…”

Section: Discussionmentioning

confidence: 99%

Time Series Anomaly Detection by Cumulative Radon Features

Hoshen¹

2022

Preprint

View full text Add to dashboard Cite

Detecting anomalous time series is key for scientific, medical and industrial tasks, but is challenging due to its inherent unsupervised nature. In recent years, progress has been made on this task by learning increasingly more complex features, often using deep neural networks. In this work, we argue that shallow features suffice when combined with distribution distance measures. Our approach models each time series as a high dimensional empirical distribution of features, where each timepoint constitutes a single sample. Modeling the distance between a test time series and the normal training set therefore requires efficiently measuring the distance between multivariate probability distributions. We show that by parameterizing each time series using cumulative Radon features, we are able to efficiently and effectively model the distribution of normal time series. Our theoretically grounded but simple-to-implement approach is evaluated on multiple datasets and shown to achieve better results than established, classical methods as well as complex, state-of-the-art deep learning methods. Code available at https: //github.com/yedidh/radonomaly

show abstract

Towards a Rigorous Evaluation of Time-series Anomaly Detection

Cited by 3 publications

References 17 publications

Intrinsic Anomaly Detection for Multi-Variate Time Series

Intrinsic Anomaly Detection for Multi-Variate Time Series

Online Time Series Anomaly Detection with State Space Gaussian Processes

Time Series Anomaly Detection by Cumulative Radon Features

Contact Info

Product

Resources

About