Duplicate Frame Video Forgery Detection Using Siamese-based RNN

Abstract

Video and image data is the most important and widely used format of communication today. It is used as evidence and authenticated proof in different domains such as law enforcement, forensic studies, journalism, and others. With the increase of video applications and data, the problem of forgery in video and images has also originated. Although a lot of work has been done on image forgery, video forensic is still a challenging area. Videos are manipulated in many ways. Frame insertion, deletion, and frame duplication are a few of the major challenges. Moreover, in the perspective of duplicated frames, frame rate variation and loop detection are also key issues. Identification of forged duplication frames for large videos with variant frame rates in real-time is not applicable due to computational limitations, lack of generalization, and low-performance accuracy. This research has investigated the problem of frame duplication with varied frame rates using a deep learning approach. A novel deep learning framework consisting of Inflated 3D (I3D) and Siamese-based Recurrent Neural Network (RNN) is proposed to resolve the aforementioned issues. The first step in the proposed framework is to extract the features and convert videos into frames. I3D network receives an original and a forged video to detect frame-to-frame duplication. Then multiple frames are merged to create a sequence. This sequence is passed to Siamese-based RNN which is used for the sequence to sequence forgery detection in video. Media Forensic Challenge (MFC) is a relatively new dataset with various frame rates, and a huge volume of videos. MFC and Image Retrieval and Analysis Tool (VIRAT) datasets are used for training and validation of the proposed model. The accuracy of the proposed method with the VIRAT dataset is 86.6% and with the MFC dataset 93%. The comparative analysis with stateof-the-art approaches has shown the robustness of the proposed approach.

Forged Video Detection Using Deep Learning: A SLR

“…Identifcation of forged duplication frames in large videos with variant frame rates in real time is not feasible due to computational limitations, lack of generalization, and low-performance accuracy [8]. Scheme based on tensor representation and orthogonal tracing feature algorithms may have limitations in detecting and locating insertion and deletion forgery in videos for more types of attacks beyond the ones tested [9].…”

“…In a variety of felds, including law enforcement, forensic research, media, and others, it is utilized as proof and verifed evidence. Te issue of video and picture counterfeiting has emerged along with the growth of video applications and data [8].…”

Real‐time object‐removal tampering localization in surveillance videos by employing YOLO‐V8

“…This approach is also limited to inter-frame fraud. Munawar and Noreen [26] developed a twin 3D RNN (recurrent neural network) to detect the duplication of video frames. This method needs higher accuracy.…”