An Attention-Based ConvLSTM Autoencoder with Dynamic Thresholding for Unsupervised Anomaly Detection in Multivariate Time Series

Abstract

As a substantial amount of multivariate time series data is being produced by the complex systems in smart manufacturing (SM), improved anomaly detection frameworks are needed to reduce the operational risks and the monitoring burden placed on the system operators. However, building such frameworks is challenging, as a sufficiently large amount of defective training data is often not available and frameworks are required to capture both the temporal and contextual dependencies across different time steps while being robust to noise. In this paper, we propose an unsupervised Attention-Based Convolutional Long Short-Term Memory (ConvLSTM) Autoencoder with Dynamic Thresholding (ACLAE-DT) framework for anomaly detection and diagnosis in multivariate time series. The framework starts by pre-processing and enriching the data, before constructing feature images to characterize the system statuses across different time steps by capturing the inter-correlations between pairs of time series. Afterwards, the constructed feature images are fed into an attention-based ConvLSTM autoencoder, which aims to encode the constructed feature images and capture the temporal behavior, followed by decoding the compressed knowledge representation to reconstruct the feature images’ input. The reconstruction errors are then computed and subjected to a statistical-based, dynamic thresholding mechanism to detect and diagnose the anomalies. Evaluation results conducted on real-life manufacturing data demonstrate the performance strengths of the proposed approach over state-of-the-art methods under different experimental settings.

Online Time-series Anomaly Detection: A Survey of Modern Model-based Approaches

“…For example, if an anomaly is detected, the channel that most likely caused the obtained model response could be highlighted to direct attention to a potential problem. So far, only Tayeh et al (2022), Zhang et al (2019), Homayouni et al (2020 and Su et al (2019) have proposed approaches that provide information about the problematic input channel. It is challenging to establish a high level of trust in data-driven models which holds back their wide-spread adoption.…”

“…The algorithm can also provide a root cause analysis by labelling the channels associated with the three worst reconstructed correlations in a given matrix. Tayeh et al (2022) proposed a similar an autoencoder structure similar to Zhang et al (2019), though without the skip connections. Here a Bahdanaustyle attention mechanism (Bahdanau et al, 2015) has been added to the model to maintain performance with increasing length of input sequences.…”

Satin Bowerbird Optimization With Convolutional LSTM for Food Crop Classification on UAV Imagery

“…Here, the food crop classification process is performed through the ConvLSTM model. LSTM-DNN model is a kind of Recurrent Neural Network (RNN) that excels in modelling temporal behaviors like | text, language, audio and time series, owing to the additional parameter metric available for the connection among time steps along with the feedback loop used for learning [25]. The main component of LSTM is forget gate, input gate, output gate and memory cells.…”

Predictive Analytics for Hydropower Fleet Intelligence

“…Another reason that we opt in for ConvLSTM is also ability to scale up the input (and output if necessary) parameter space without causing computational overload as much as other methods that utilize covariance matrix such as Gaussian Process. There are multiple applications in the literature that use ConvLSTM for multi-variate timeseries prediction (Xiao et al, 2021;Mu et al, 2019;Tayeh et al, 2022). In the future, we are aiming to include many other relevant sensor variables (temperature, current, etc.)…”