Learning Memory-Guided Normality for Anomaly Detection

Park, Hyunjong; Noh, Jongyoun; Ham, Bumsub

doi:10.1109/cvpr42600.2020.01438

Cited by 662 publications

(467 citation statements)

References 31 publications

Supporting

Mentioning

461

Contrasting

Unclassified

Order By: Relevance

“…By changing the threshold gradually, we can arrive at a ROC curve. The above methods learn normal feature pattern [23,24], normal frame prediction [21,26], and normal skeleton pattern [13,14]. As abnormal video has not been seen in training, therefore, the unsupervised methods get lower performance than the supervised methods.…”

Section: Comparison With Unsupervised Methodsmentioning

confidence: 99%

Deep social force network for anomaly event detection

Yang

Wang

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Anomaly event detection is vital in surveillance video analysis. However, how to learn the discriminative motion in the crowd scene is still not tackled. Here, a deep social force network by exploiting both social force extracting and deep motion coding is proposed. Given a grid of particles with velocity provided by the optical flow, the interaction force in the crowd scene is investigated and a social force module is embedded in a deep network. A deep motion convolution was further designed with a 3D (DMC-3D) module. The DMC-3D not only eliminates the noise motion in the crowd scene with a spatial encoder-decoder but also learns the 3D feature with a spatio-temporal encoder. The deep social force coding is modelled with multiple features, in which each feature can describe specific anomaly motion. The experiments on UCF-Crime and ShanghaiTech datasets demonstrate that our method can predict the temporal localization of anomaly events and outperform the stateof-the-art methods.

show abstract

Section: Comparison With Unsupervised Methodsmentioning

confidence: 99%

Deep social force network for anomaly event detection

Yang

Wang

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…In the following work, Miller et al (2016) improves the efficiency of retrieving large memories by pre-selecting small subsets with key hashing. Moreover, the memory network is further applied in video analysis ( Shi et al, 2019 , Park et al, 2020 , Lai et al, 2020 ) and image captioning ( Cornia et al, 2020 ). In Shi et al (2019) , the authors devise a dual augmented memory network to memorize both target and background features of an video, and use a Long Short-Term Memory (LSTM) to communicate with previous and next frames.…”

Section: Related Workmentioning

confidence: 99%

“…In Shi et al (2019) , the authors devise a dual augmented memory network to memorize both target and background features of an video, and use a Long Short-Term Memory (LSTM) to communicate with previous and next frames. In Park et al (2020) , the authors propose a memory network to memorize normal patterns for detecting anomalies in an video. As an attempt in image captioning, Cornia et al (2020) devise a learnable memory to learn and memorize priori knowledge for encoding relationships between image regions.…”

Section: Related Workmentioning

confidence: 99%

“…We believe that this learning process also applies to the interpretation of multi-scene aerial images. Driven by this observation, we propose a novel network, termed as prototype-based memory network (PM-Net), which is inspired by recent successes of memory networks in natural language processing (NLP) tasks ( Sukhbaatar et al, 2015 , Miller et al, 2016 ) and video analysis ( Shi et al, 2019 , Park et al, 2020 , Lai et al, 2020 ). To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial images and then store these prototypes in the external memory of PM-Net.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks

Hua

Mou

Lin

et al. 2021

ISPRS Journal of Photogrammetry and Remote Sensing

View full text Add to dashboard Cite

Aerial scene recognition is a fundamental visual task and has attracted an increasing research interest in the last few years. Most of current researches mainly deploy efforts to categorize an aerial image into one scene-level label, while in real-world scenarios, there often exist multiple scenes in a single image. Therefore, in this paper, we propose to take a step forward to a more practical and challenging task, namely multi-scene recognition in single images. Moreover, we note that manually yielding annotations for such a task is extraordinarily time- and labor-consuming. To address this, we propose a prototype-based memory network to recognize multiple scenes in a single image by leveraging massive well-annotated single-scene images. The proposed network consists of three key components: 1) a prototype learning module, 2) a prototype-inhabiting external memory, and 3) a multi-head attention-based memory retrieval module. To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory. Afterwards, a multi-head attention-based memory retrieval module is devised to retrieve scene prototypes relevant to query multi-scene images for final predictions. Notably, only a limited number of annotated multi-scene images are needed in the training phase. To facilitate the progress of aerial scene recognition, we produce a new multi-scene aerial image (MAI) dataset. Experimental results on variant dataset configurations demonstrate the effectiveness of our network. Our dataset and codes are publicly available 1 .

show abstract

“…Therefore, one of the main research issues is how to generate a reference image. One common method is to utilize generative models [1]- [14]. This method aims to generate a reference image by transforming abnormal patterns of a test image, if any, into normal patterns observed in normal training images.…”

Section: Introductionmentioning

confidence: 99%

Patch-Level Operation With Adaptive Patch Control for Improving Anomaly Localization

et al. 2021

View full text Add to dashboard Cite

In realizing unsupervised pixel-precise anomaly localization by utilizing a generative model, a reference image must be generated (for comparison with an input image) by transforming abnormal patterns of an input image, if any, into normal patterns. In this study, a patch-level operation with adaptive patch control is proposed to improve anomaly localization by generating a better reference image. As a way to exploit a generative model, we divide an image into non-overlapped patches of the same size, generate patch-level reference images, and stitch the patch-level reference images into a single reference image. We then conduct anomaly localization by comparing an input image with the stitched, reconstructed image. To effectively apply the patch-level operation, we propose adaptive patch control to determine the number of non-overlapped patches to be applied. For this, we synthesize defective images using normal images and examine how well the candidate methods with different numbers of patches remove the synthesized defects. In the same way, we utilize adaptive patch control to select a promising model among the candidate generative models. Based on experiments conducted using the MVTec Anomaly Detection dataset, we demonstrate that our method outperforms previous existing methods. Under a real-world scenario, our method shows ROC AUC of 0.926, in contrast to the best value of 0.893 from existing studies. Furthermore, we prove the feasibility of the adaptive patch control by showing that the removal of the synthesized defects and the anomaly localization for real defective images are highly correlated.

show abstract

Learning Memory-Guided Normality for Anomaly Detection

Cited by 662 publications

References 31 publications

Deep social force network for anomaly event detection

Deep social force network for anomaly event detection

Aerial scene understanding in the wild: Multi-scene recognition via prototype-based memory networks

Patch-Level Operation With Adaptive Patch Control for Improving Anomaly Localization

Contact Info

Product

Resources

About