Deep Learning for Automatic Violence Detection: Tests on the AIRTLab Dataset

Abstract

Following the growing availability of video surveillance cameras and the need for techniques to automatically identify events in video footages, there is an increasing interest towards automatic violence detection in videos. Deep learning-based architectures, such as 3D Convolutional Neural Networks, demonstrated their capability of extracting spatio-temporal features from videos, being effective in violence detection. However, friendly behaviours or fast moves such as hugs, small hits, claps, high fives, etc., can still cause false positives, interpreting a harmless action as violent. To this end, we present three deep-learning based models for violence detection and test them on the AIRTLab dataset, a novel dataset designed to check the robustness of algorithms against false positives. The objective is twofold: on one hand, we compute accuracy metrics on the three proposed models (two are based on transfer learning and one is trained from scratch), building a baseline of metrics for the AIRTLab dataset; on the other hand, we validate the capability of the proposed dataset of challenging the robustness to false positives. The results of the proposed models are in line with the scientific literature, in terms of accuracy, with transfer learning-based networks exhibiting better generalization capabilities than the trained from scratch network. Moreover, the tests highlighted that most of the classification errors concern the identification of non-violent clips, validating the design of the proposed dataset. Finally, to demonstrate the significance of the proposed models, the paper presents a comparison with the related literature, as well as with models based on well-established pre-trained 2D Convolutional Neural Networks 2D CNNs. Such comparison highlights that 3D models get better accuracy performance than time distributed 2D CNNs (merged with a recurrent model) in processing the spatiotemporal features of video clips. The source code of the experiments and the AIRTLab dataset are available in public repositories.

Video Violence Detection Using LSTM and Transformer Networks Through Grid Search-Based Hyperparameters Optimization

“…ROC curves for Hockey (top row), Crowd (second row), and AIRTLab (third row) are extracted from Fold 5, Fold 3, and Fold 2, respectively. ViF [4] 0.8801 OViF [5] 0.9193 DiMOLIF [29] 0.9323 LBP+GLCM [3] 0.9360 HOMO [7] 0.9518 3D CNN [1] 0.970 MoWLD [6] 0.9758 LHOG+LHOF [8] 0.9798 C3D+SVM [20] 0.9962 C3D+FC [20] 0.9927 ConvLSTM [20] 0.9931 Ours (Transformer) 0.954 Ours (LSTM) 0.976 Crowd HOMO [7] 0.8284 ViF [4] 0.8804 DiMOLIF [29] 0.8925 OViF [5] 0.9182 LBP+GLCM [3] 0.93 MoWLD [6] 0.9408 ConvLSTM [20] 0.9443 LHOG+LHOF [8] 0.9703 3D CNN [1] 0.98 C3D+FC [20] 0.9994 C3D+SVM [20] 1 Ours (Transformer) 0.916 Ours (LSTM) 0.934 AIRTLab C3D+SVM [20] 0.993 C3D+FC [20] 0.9894 ConvLSTM [20] 0.9967 Ours (Transformer) 0.79 Ours (LSTM) 0.86…”

“…MoWLD outperforms our LSTM, but only marginally. The top models in Hockey, Crowd, and AIRTLab are still C3D [20] and ConvLSTM [20] according to the table.…”

“…Since the number of initialized hyperparameter values expands along with the computing of Grid Search, we tracked the value of a few hyperparameters based on prior research. In certain investigations, the dropout probability was chosen at between 0.2 and 0.5 [20,[26][27][28][29]. The learning rates are referred to Ullah's work [1], which is set at 0.0001 and 0.00001.…”

Toward Fast and Accurate Violence Detection for Automated Video Surveillance Applications

“…In their study, P. Sernani et al [97] proposed the AIRTLab dataset, which contains videos showing violence patterns performed by non-professional actors. They studied the use of 2D and 3D deep learning architectures for violence detection using their dataset and found that the studied models adapt well to their setting, where violence is mimicked by non-professional actors.…”

Bus Violence: An Open Benchmark for Video Violence Detection on Public Transport

“…More recently, the authors of [ 19 , 20 ] proposed the AIRTLab dataset, a small collection of 350 video clips labeled as “non-violent” and “violent,” where the non-violent actions include behaviors such as hugs and claps that can cause false positives in the violence detection task. Furthermore, the Surveillance Camera Fight dataset has been presented in [ 21 ].…”