Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Yu, Guang; Wang, Siqi; Cai, Zhiping; Zhu, En; Xu, Chuanfu; Yin, Jianping; Kloft, Marius

doi:10.1145/3394171.3413973

Cited by 170 publications

(110 citation statements)

References 44 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…Reconstruction of masked information. A surrogate task for many anomaly detection approaches [15,22,37,42,77] is to erase some information from the input, while making neural networks predict the erased information. Haselmann et al [22] framed anomaly detection as an inpainting problem, where patches from images are masked randomly, using the pixel-wise reconstruction error of the masked patches for surface anomaly detection.…”

Section: Related Workmentioning

confidence: 99%

“…In turn, ARNet learns to restore the original image and detect anomalies based on the assumption that normal images can be restored properly. The Cloze task [42] learns to complete a video when certain frames are removed, being recently employed by Yu et al [77] for anomaly detection. In a similar direction, Georgescu et al [17] proposed middle frame masking as one of the auxiliary tasks for video anomaly detection.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection

Ristea¹,

Madan²,

Ionescu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Anomaly detection is commonly pursued as a one-class classification problem, where models can only learn from normal training samples, while being evaluated on both normal and abnormal test samples. Among the successful approaches for anomaly detection, a distinguished category of methods relies on predicting masked information (e.g. patches, future frames, etc.) and leveraging the reconstruction error with respect to the masked information as an abnormality score. Different from related methods, we propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. The proposed self-supervised block is generic and can easily be incorporated into various state-of-the-art anomaly detection methods. Our block starts with a convolutional layer with dilated filters, where the center area of the receptive field is masked. The resulting activation maps are passed through a channel attention module. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video, providing empirical evidence that shows considerable performance improvements on MVTec AD, Avenue, and ShanghaiTech.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection

Ristea¹,

Madan²,

Ionescu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast, our approach encourages AEs to produce unconstrained reconstructions for normal inputs while limiting the reconstructions for anomalous inputs, thus producing more discriminative anomaly scores. Non-Reconstruction Methods: Several researchers adopt different schemes for OCC based anomaly detection: focusing only on objects by utilizing object detectors in the frameworks [6,7,8,11,12,43,52]; predicting future frames from the past few consecutive frames with the intuition that it is difficult to predict unseen anomalous data [5,24,27,28,35]; or incorporating adversarial components [14,19,20,24,39,45]. Our approach is different as we do not utilize any additional component and solely rely on the reconstruction based AEs.…”

Section: Related Workmentioning

confidence: 99%

Learning Not to Reconstruct Anomalies

Astrid¹,

Zaheer²,

Lee³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video anomaly detection is often seen as one-class classification (OCC) problem due to the limited availability of anomaly examples. Typically, to tackle this problem, an autoencoder (AE) is trained to reconstruct the input with training set consisting only of normal data. At test time, the AE is then expected to well reconstruct the normal data while poorly reconstructing the anomalous data. However, several studies have shown that, even with only normal data training, AEs can often start reconstructing anomalies as well which depletes the anomaly detection performance. To mitigate this problem, we propose a novel methodology to train AEs with the objective of reconstructing only normal data, regardless of the input (i.e., normal or abnormal). Since no real anomalies are available in the OCC settings, the training is assisted by pseudo anomalies that are generated by manipulating normal data to simulate the out-of-normal-data distribution. We additionally propose two ways to generate pseudo anomalies: patch and skip frame based. Extensive experiments on three challenging video anomaly datasets demonstrate the effectiveness of our method in improving conventional AEs, achieving state-of-theart performance.

show abstract

“…In spite of the growing interest in video anomaly detection [9, 10, 14-16, 19-21, 24, 29, 31, 36-38, 40, 43, 49, 51, 57, 58, 61, 63], which generated significant advances leading to impressive performance levels [14,15,18,24,29,53,56,57,61,63,64], the task remains very challenging. The difficulty of the task stems from two interdependent aspects: (i) the reliance on context of anomalies, and (ii) the lack of abnormal training data.…”

Section: Introductionmentioning

confidence: 99%

UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection

Acsintoae¹,

Florescu²,

Mare³

et al. 2021

Preprint

View full text Add to dashboard Cite

Detecting abnormal events in video is commonly framed as a one-class classification task, where training videos contain only normal events, while test videos encompass both normal and abnormal events. In this scenario, anomaly detection is an open-set problem. However, some studies assimilate anomaly detection to action recognition. This is a closed-set scenario that fails to test the capability of systems at detecting new anomaly types. To this end, we propose UBnormal, a new supervised open-set benchmark composed of multiple virtual scenes for video anomaly detection. Unlike existing data sets, we introduce abnormal events annotated at the pixel level at training time, for the first time enabling the use of fully-supervised learning methods for abnormal event detection. To preserve the typical open-set formulation, we make sure to include disjoint sets of anomaly types in our training and test collections of videos. To our knowledge, UBnormal is the first video anomaly detection benchmark to allow a fair headto-head comparison between one-class open-set models and supervised closed-set models, as shown in our experiments. Moreover, we provide empirical evidence showing that UBnormal can enhance the performance of a state-of-the-art anomaly detection framework on two prominent data sets, Avenue and ShanghaiTech.

show abstract

Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

Cited by 170 publications

References 44 publications

Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection

Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection

Learning Not to Reconstruct Anomalies

UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection

Contact Info

Product

Resources

About