Integrated Multiscale Appearance Features and Motion Information Prediction Network for Anomaly Detection

Abstract

The rise of video-prediction algorithms has largely promoted the development of anomaly detection in video surveillance for smart cities and public security. However, most current methods relied on single-scale information to extract appearance (spatial) features and lacked motion (temporal) continuity between video frames. This can cause a loss of partial spatiotemporal information that has great potential to predict future frames, affecting the accuracy of abnormality detection. Thus, we propose a novel prediction network to improve the performance of anomaly detection. Due to the objects of various scales in each video, we use different receptive fields to extract detailed appearance features by the hybrid dilated convolution (HDC) module. Meanwhile, the deeper bidirectional convolutional long short-term memory (DB-ConvLSTM) module can remember the motion information between consecutive frames. Furthermore, we use RGB difference loss to replace optical flow loss as temporal constraint, which greatly reduces the time for optical flow extraction. Compared with the state-of-the-art methods in the anomaly-detection task, experiments prove that our method can more accurately detect abnormalities in various video surveillance scenes.

A Hybrid Approach to Improve the Video Anomaly Detection Performance of Pixel- and Frame-Based Techniques Using Machine Learning Algorithms

“…Other than these, different methods have been used for human activity recognition [71,72], video object detection [73][74][75], video motion anomaly detection [76][77][78][79], online anomaly detection [80,81], video anomaly detection by injecting temporal information in feature extraction [82], anomaly detection method with deep support vector data description (DSVDD) using deep learning algorithm [83], video anomaly detection method with a main-auxiliary aggregation strategy (MAAS) [84], and the analysis of anomalies with feature embeddings of pre-trained CNNs with the use of novel cross-domain generalization measures [85] in various studies.…”