Dance with Self-Attention: A New Look of Conditional Random Fields on Anomaly Detection in Videos

Purwanto, Didik; Chen, Yie‐Tarng; Fang, Wen‐Hsien

doi:10.1109/iccv48922.2021.00024

Cited by 39 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Weakly-supervised models use a small number of abnormal frames with labels for training. Based on the triplet loss [18] or multiple instance learning strategy [40,54,53,7,44,35], such models can learn to increase the inter-class distance between normal and abnormal data. Weakly-supervised models outperform unsupervised models but they require abnormal data with ground truth labels [50,7,44].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Look at Adjacent Frames: Video Anomaly Detection without Offline Training

Ouyang¹,

Shen²,

Sánchez³

2022

Preprint

View full text Add to dashboard Cite

We propose a solution to detect anomalous events in videos without the need to train a model offline. Specifically, our solution is based on a randomly-initialized multilayer perceptron that is optimized online to reconstruct video frames, pixel-by-pixel, from their frequency information. Based on the information shifts between adjacent frames, an incremental learner is used to update parameters of the multilayer perceptron after observing each frame, thus allowing to detect anomalous events along the video stream. Traditional solutions that require no offline training are limited to operating on videos with only a few abnormal frames. Our solution breaks this limit and achieves strong performance on benchmark datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Despite the fact that the performance of VAD solutions has improved recently [9,35], online VAD solutions are still scarce [28,36]. Moreover, the existing ones are usually flawed.…”

Section: Introductionmentioning

confidence: 99%

Look at Adjacent Frames: Video Anomaly Detection without Offline Training

Ouyang¹,

Shen²,

Sánchez³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To gain a more discriminative model utilizing local and global features and their short-range correlations, Purwanto et al [19] integrate a self-attention module containing conditional random fields (CRFs) with CNN. The relation-aware feature extractor extending the temporal relational network (TRN) is also included to extract multiscale CNN features.…”

Section: Weakly Supervised Learningmentioning

confidence: 99%

“…AnoGAN [9] is a typical generative method and EBGAN [10] borrows the concept of energy. Clustering is a widely used technique [11,18,19] to push the anomalies away from normal instances and minimize the distance between normal instances. To use features of neighboring frames, some proposed the Recurrent Neural Network (RNN)-based model to store key information in the model even if taking images as input.…”

Section: Introductionmentioning

confidence: 99%

Weakly-Supervised and Unsupervised Video Anomaly Detection

Song¹

2022

HSET

View full text Add to dashboard Cite

As surveillance technology is continuously improving, an ever-increasing number of cameras are being deployed everywhere. Relying on manual detection of anomalies through cameras may be unreliable and untimely. Therefore, the application of deep learning in video anomaly detection is being extensively studied. Anomaly Detection (AD) refers to identifying events that deviate from the desired actions. This article discusses representative unsupervised and weakly-supervised learning methods applied to various data types. In these machine learning methods, Generative Adversarial Network, Auto Encoder, Recurrent Neural Network, etc. are broadly adopted for AD. Some renowned and new datasets are reviewed. Furthermore, we also proposed several future directions of research in video anomaly detection.

show abstract

“…Due its applicability in video surveillance, anomaly detection is an actively studied topic in the video domain, with many recent attempts trying to solve the problem by employing various approaches ranging from outlier detection * corresponding author: raducu.ionescu@gmail.com models [2,8,9,11,15,21,29,32,35,37,40,43,45,47,48,52,54,55,56,58,59,61,64,69,75,78,87,85,86] and weakly-supervised learning frameworks [16,53,67,70,83,88] to supervised open-set methods [1]. Despite the numerous attempts in solving the problem, video anomaly detection remains a challenging task, especially due to the fact that abnormal events are determined by the context.…”

Section: Introductionmentioning

confidence: 99%

SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection

Barbalau¹,

Ionescu²,

Dueholm³

et al. 2022

Preprint

View full text Add to dashboard Cite

A self-supervised multi-task learning (SSMTL) framework for video anomaly detection was recently introduced in literature. Due to its highly accurate results, the method attracted the attention of many researchers. In this work, we revisit the self-supervised multi-task learning framework, proposing several updates to the original method. First, we study various detection methods, e.g. based on detecting high-motion regions using optical flow or background subtraction, since we believe the currently used pre-trained YOLOv3 is suboptimal, e.g. objects in motion or objects from unknown classes are never detected. Second, we modernize the 3D convolutional backbone by introducing multihead self-attention modules, inspired by the recent success of vision transformers. As such, we alternatively introduce both 2D and 3D convolutional vision transformer (CvT) blocks. Third, in our attempt to further improve the model, we study additional self-supervised learning tasks, such as predicting segmentation maps through knowledge distillation, solving jigsaw puzzles, estimating body pose through knowledge distillation, predicting masked regions (inpainting), and adversarial learning with pseudo-anomalies. We conduct experiments to assess the performance impact of the introduced changes. Upon finding more promising configurations of the framework, dubbed SSMTL++v1 and SSMTL++v2, we extend our preliminary experiments to more data sets, demonstrating that our performance gains are consistent across all data sets. In most cases, our results on Avenue, ShanghaiTech and UBnormal raise the state-ofthe-art performance to a new level.

show abstract

Dance with Self-Attention: A New Look of Conditional Random Fields on Anomaly Detection in Videos

Cited by 39 publications

References 35 publications

Look at Adjacent Frames: Video Anomaly Detection without Offline Training

Look at Adjacent Frames: Video Anomaly Detection without Offline Training

Weakly-Supervised and Unsupervised Video Anomaly Detection

SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection

Contact Info

Product

Resources

About