An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos

Kiran, Bangalore Ravi; Thomas, Dilip Mathew; Parakkal, Ranjith

doi:10.3390/jimaging4020036

Cited by 400 publications

(177 citation statements)

References 65 publications

Supporting

Mentioning

176

Contrasting

Order By: Relevance

“…Image resynthesis and generation methods, such as autoencoder and GANs, have been used in the past for anomaly detection. The existing methods, however, mostly focus on finding behavioral anomalies in the temporal domain [36,21]. For example, [36] predicts the optical flow in a video, attempts to reconstruct the images from the flow, and treats significant differences from the original images as evidence for an anomaly.…”

Section: Anomaly Detection Via Resynthesismentioning

confidence: 99%

Detecting the Unexpected via Image Resynthesis

Lis¹,

Nakka

Fua

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

147

139

View full text Add to dashboard Cite

Classical semantic segmentation methods, including the recent deep learning ones, assume that all classes observed at test time have been seen during training. In this paper, we tackle the more realistic scenario where unexpected objects of unknown classes can appear at test time. The main trends in this area either leverage the notion of prediction uncertainty to flag the regions with low confidence as unknown, or rely on autoencoders and highlight poorly-decoded regions. Having observed that, in both cases, the detected regions typically do not correspond to unexpected objects, in this paper, we introduce a drastically different strategy: It relies on the intuition that the network will produce spurious labels in regions depicting unexpected objects. Therefore, resynthesizing the image from the resulting semantic map will yield significant appearance differences with respect to the input image. In other words, we translate the problem of detecting unknown classes to one of identifying poorlyresynthesized image regions. We show that this outperforms both uncertainty-and autoencoder-based methods.

show abstract

Section: Anomaly Detection Via Resynthesismentioning

confidence: 99%

Detecting the Unexpected via Image Resynthesis

Lis¹,

Nakka

Fua

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

147

139

View full text Add to dashboard Cite

show abstract

“…Ideally, anomaly detection should not build upon case-specific assumptions in the form of medical domain knowledge or specific annotated validation sets to optimize for, which should be interpreted as an unwanted form of supervision implicitly added by design of the method. Variational Auto-Encoders (VAEs) and their extensions are, alongside flow-based and auto-regressive models, a current de-facto standard for density estimation and particularly anomaly/out-of-distribution sample detection tasks [1,3,12,14]. Here, the evidence lower bound (ELBO), by definition a combination of the reconstruction error with the Kullback-Leibler (KL)-divergence, commonly serves as a proxy for the sample likelihood [3,12].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Anomaly Localization Using Variational Auto-Encoders

Zabala-Blanco

Isensee

Petersen

et al. 2019

Lecture Notes in Computer Science

100

View full text Add to dashboard Cite

An assumption-free automatic check of medical images for potentially overseen anomalies would be a valuable assistance for a radiologist. Deep learning and especially Variational Auto-Encoders (VAEs) have shown great potential in the unsupervised learning of data distributions. In principle, this allows for such a check and even the localization of parts in the image that are most suspicious. Currently, however, the reconstruction-based localization by design requires adjusting the model architecture to the specific problem looked at during evaluation. This contradicts the principle of building assumption-free models. We propose complementing the localization part with a term derived from the Kullback-Leibler (KL)-divergence. For validation, we perform a series of experiments on FashionMNIST as well as on a medical task including >1000 healthy and >250 brain tumor patients. Results show that the proposed formalism outperforms the state of the art VAE-based localization of anomalies across many hyperparameter settings and also shows a competitive max performance.

show abstract

“…According to [9] deep learning based anomaly detection methods can be divided into three different categories: representation learning for reconstruction, predictive modeling, and generative models. Methods categorized as representation learning for reconstruction are used to find a transformation of the training data which defines the normal behavior.…”

Section: Overview Of Anomaly Detection Methodsmentioning

confidence: 99%

What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks

Golda¹,

Murzyn²,

Qu³

et al. 2019

2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)

View full text Add to dashboard Cite

Anomaly detection plays in many fields of research, along with the strongly related task of outlier detection, a very important role. Especially within the context of the automated analysis of video material recorded by surveillance cameras, abnormal situations can be of very different nature. For this purpose this work investigates Generative-Adversarial-Network-based methods (GAN) for anomaly detection related to surveillance applications. The focus is on the usage of static camera setups, since this kind of camera is one of the most often used and belongs to the lower price segment. In order to address this task, multiple subtasks are evaluated, including the influence of existing optical flow methods for the incorporation of short-term temporal information, different forms of network setups and losses for GANs, and the use of morphological operations for further performance improvement. With these extension we achieved up to 2.4% better results. Furthermore, the final method reduced the anomaly detection error for GANbased methods by about 42.8%.

show abstract

An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos

Cited by 400 publications

References 65 publications

Detecting the Unexpected via Image Resynthesis

Detecting the Unexpected via Image Resynthesis

Unsupervised Anomaly Localization Using Variational Auto-Encoders

What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks

Contact Info

Product

Resources

About