Flow accumulation-based violence detection model using transformers

Abstract

Violence detection in surveillance videos is one of the most challenging tasks in the field of video analysis. This work aims to enhance the accuracy and efficiency of the violence detection model by leveraging optical flow accumulation alongside traditional RGB frames as input to a transformer-based model. A novel frame extraction method is also proposed which accumulates motion information in a video. The flow-accumulated frames and the original frames of the video are combined in the Flow Fusion Attention Block. The proposed approach serves as a solution to increase the classification accuracy of transformer models in violent detection. The model achieves high accuracy by combining the RGB and flow-accumulated frames through the flow fusion attention block. The proposed approach is evaluated using different datasets and the performance is better than the other state-of-the-art approaches for violence detection.