Physics-Based Anomaly Detection Defined on Manifold Space

Abstract

Current popular anomaly detection algorithms are capable of detecting global anomalies but often fail to distinguish local anomalies from normal instances. Inspired by contemporary physics theory (i.e., heat diffusion and quantum mechanics), we propose two unsupervised anomaly detection algorithms. Building on the embedding manifold derived from heat diffusion, we devise Local Anomaly Descriptor (LAD), which faithfully reveals the intrinsic neighborhood density. It uses a scale-dependent umbrella operator to bridge global and local properties, which makes LAD more informative within an adaptive scope of neighborhood. To offer more stability of local density measurement on scaling parameter tuning, we formulate Fermi Density Descriptor (FDD), which measures the probability of a fermion particle being at a specific location. By choosing the stable energy distribution function, FDD steadily distinguishes anomalies from normal instances with any scaling parameter setting. To further enhance the efficacy of our proposed algorithms, we explore the utility of anisotropic Gaussian kernel (AGK), which offers better manifold-aware affinity information. We also quantify and examine the effect of different Laplacian normalizations for anomaly detection. Comprehensive experiments on both synthetic and benchmark datasets verify that our proposed algorithms outperform the existing anomaly detection algorithms.

AdaGUM: An Adaptive Graph Updating Model-Based Anomaly Detection Method for Edge Computing Environment

“…e basic principle of anomaly detection is to detect the data which are deviated from the known normal patterns according to a predefined similarity threshold. Currently, the common anomaly detection methods include the anomaly detection methods based on distance [14,15], the anomaly detection methods based on density [16,17], the anomaly detection methods based on classification [18,19], and the anomaly detection methods based on models [20][21][22]. With the development of machine learning, lots of advanced algorithms and models have been proposed successively and integrated with the existing traditional anomaly detection methods in order to improve their detection performance on both accuracy and efficiency [23][24][25], such as the detection methods based on the neural network model, the detection methods based on k nearest neighbor (k-NN), and the detection methods based on the support vector machine (SVM).…”

An adaptive method based on contextual anomaly detection in Internet of Things through wireless sensor networks

“…However, the performance of these SVM-based anomaly detection methods suffer from the sensitivity to missing data. Huang et al 37 proposed two unsupervised anomaly detection algorithms which can distinguish local anomalies from normal instances. But it neglects the changes of anomalous behaviors.…”

Computational reproducibility of scientific workflows at extreme scales

“…One method that we are exploring to reduce the size of performance data is to use change detection algorithms with in situ monitoring to store only events of interest for performance optimization of scientific workflows. We consider the extension of streaming manifold learning (Yoo et al, 2016) with anomaly score calculation (Huang et al, 2014) on the performance data. Current data reduction techniques within MD calculations rely on a mechanical sampling strategy, storing 1 out of 100 to 1000 time steps from a running simulation, thus possibly discarding meaningful structures from the output.…”