Multi-Level Two-Stream Fusion-Based Spatio-Temporal Attention Model for Violence Detection and Localization

Abstract

Detection of violent human behavior is necessary for public safety and monitoring. However, it demands constant human observation and attention in human-based surveillance systems, which is a challenging task. Autonomous detection of violent human behavior is therefore essential for continuous uninterrupted video surveillance. In this paper, we propose a novel method for violence detection and localization in videos using the fusion of spatio-temporal features and attention model. The model consists of Fusion Convolutional Neural Network (Fusion-CNN), spatio-temporal attention modules and Bi-directional Convolutional LSTMs (BiConvLSTM). The Fusion-CNN learns both spatial and temporal features by combining multi-level inter-layer features from both RGB and Optical flow input frames. The spatial attention module is used to generate an importance mask to focus on the most important areas of the image frame. The temporal attention part, which is based on BiConvLSTM, identifies the most significant video frames which are related to violent activity. The proposed model can also localize and discriminate prominent regions in both spatial and temporal domains, given the weakly supervised training with only video-level classification labels. Experimental results evaluated on different publicly available benchmarking datasets show the superior performance of the proposed model in comparison with the existing methods. Our model achieves the improved accuracies (ACC) of 89.1%, 99.1% and 98.15% for RWF-2000, HockeyFight and Crowd-Violence datasets, respectively. For CCTV-FIGHTS dataset, we choose the mean average precision (mAp) performance metric and our model obtained 80.7% mAp.

Home Activity Recognition for Rural Elderly Based on Deep Learning and Smartphone Sensors

“…4. Spatiotemporal attention network (Asad et al, 2022): This network integrates spatial and temporal information and introduces an attention mechanism to focus on crucial spatiotemporal regions in video sequences. In elderly behavior detection, the spatiotemporal attention network effectively captures spatial and temporal behavior patterns.…”

Research on Local Counting and Object Detection of Multiscale Crowds in Video Based on Time-Frequency Analysis

“…Our work improves method [63] in that we both train a detection network for crowd counting. While training fully supervised detectors with bounding box annotations, we only train weakly supervised detectors with pointlevel annotations [64]. Unlike our method, which only focuses on the number of people, our goal is to predict the number of people and generate appropriately sized detection boxes.…”

The Modern Electronic and other Technologies to Combat New Wave of Terrorism and Criminal Activities

“…Department of Automation at Shanghai Jiao Tong University in Shanghai, China, created monitoring and surveillance technologies that rely entirely on automated detection and analysis. Based on "Multi-Stream 3D latent feature clustering for abnormality detection in videos" connected with CCTV at key locations, the full systems have been published and documented in [2] and [3]. The "Multi-Level Two Stream Fusion based Spatio-temporal Attention Model for Violence Detection and Localization" is used by the second system.…”