Video Anomaly Detection via Prediction Network with Enhanced Spatio-Temporal Memory Exchange

Abstract

Video anomaly detection is a challenging task because most anomalies are scarce and non-deterministic. Many approaches investigate the reconstruction difference between normal and abnormal patterns, but neglect that anomalies do not necessarily correspond to large reconstruction errors. To address this issue, we design a Convolutional LSTM Auto-Encoder prediction framework with enhanced spatiotemporal memory exchange using bi-directionalilty and a higher-order mechanism. The bi-directional structure promotes learning the temporal regularity through forward and backward predictions. The unique higher-order mechanism further strengthens spatial information interaction between the encoder and the decoder. Considering the limited receptive fields in Convolutional LSTMs, we also introduce an attention module to highlight informative features for prediction. Anomalies are eventually identified by comparing the frames with their corresponding predictions. Evaluations on three popular benchmarks show that our framework outperforms most existing prediction-based anomaly detection methods.

Look at Adjacent Frames: Video Anomaly Detection without Offline Training

“…Unsupervised models exclusively employ normal videos for training, with data mapping via an encoder-decoder structure being the most frequently used strategy. The baseline approaches in this context are based on data reconstruction or prediction, where anomalous events are detected based on the reconstruction or prediction errors, respectively [12,52,24,29,49,21,38]. Adversarial losses can also be applied in such approaches, where the generator is adversarially trained to perform the data mapping so that abnormal videos lead to unrealistic outputs with large reconstruction or prediction errors [37,19,45,30,51].…”

Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism

“…Shen et al. [48] designed a ConvLSTM prediction framework for AE with enhanced spatio‐temporal memory exchange using bi‐directionality and a higher‐order mechanisms. Chang et al.…”

“…Hao et al [28] proposed a spatiotemporal consistency-enhanced network to generate spatiotemporal consistency predictions. Shen et al [48] designed a ConvLSTM prediction framework for AE with enhanced spatio-temporal memory exchange using bi-directionality and a higher-order mechanisms. Chang et al [29] explore a novel convolution AE architecture that can dissociate the spatio-temporal representation to separately capture the spatial and the temporal information, and exploit a variance-based attention module and insert it into the motion AE to highlight large movement areas.…”