Spatio-temporal co-attention fusion network for video splicing localization

Abstract

Digital video splicing has become easy and ubiquitous. Malicious users copy some regions of a video and paste them into another video to create realistic forgeries. It is important to blindly detect such forgery regions in videos. A spatio-temporal co-attention fusion network (SCFNet) is proposed for video splicing localization. Specifically, a three-stream network is used as an encoder to capture manipulation traces across multiple frames. The deep interaction and fusion of spatio-temporal forensic features are achieved by the novel parallel and cross co-attention fusion modules. A lightweight multilayer perceptron decoder is adopted to yield a pixel-level tampering localization map. A new large-scale video splicing dataset is created for training the SCFNet. Extensive tests on benchmark datasets show that the localization and generalization performances of our SCFNet outperform the state-of-theart. Code and datasets are available at https://github.com/multimediaFor/SCFNet.