Deepfake video detection: YOLO-Face convolution recurrent approach

Abstract

Recently, the deepfake techniques for swapping faces have been spreading, allowing easy creation of hyper-realistic fake videos. Detecting the authenticity of a video has become increasingly critical because of the potential negative impact on the world. Here, a new project is introduced; You Only Look Once Convolution Recurrent Neural Networks (YOLO-CRNNs), to detect deepfake videos. The YOLO-Face detector detects face regions from each frame in the video, whereas a fine-tuned EfficientNet-B5 is used to extract the spatial features of these faces. These features are fed as a batch of input sequences into a Bidirectional Long Short-Term Memory (Bi-LSTM), to extract the temporal features. The new scheme is then evaluated on a new large-scale dataset; CelebDF-FaceForencics++ (c23), based on a combination of two popular datasets; FaceForencies++ (c23) and Celeb-DF. It achieves an Area Under the Receiver Operating Characteristic Curve (AUROC) 89.35% score, 89.38% accuracy, 83.15% recall, 85.55% precision, and 84.33% F1-measure for pasting data approach. The experimental analysis approves the superiority of the proposed method compared to the state-of-the-art methods.

An integrated spatiotemporal-based methodology for deepfake detection

“…As seen from Table 4, the YF method scored the highest performance. It recorded an AUROC score of 95.53% which exceeds that of other current detection methods [22,23,46,52] with an average increase of 7.695%. In addition, the running time is recorded in Table 4.…”

“…(c23). This helped to assess the robustness of the proposed fake detection method and boosted the applicability of the proposed method in the real world [22,23].…”

“…Then, a fine-tuned Incep-tionResNetV2 model was employed as a feature extractor, followed by the XGBoost model as a classifier to distinguish a deepfake video from a genuine one. In [23], a method to detect the spatiotemporal discrepancies in deepfake videos was introduced. This method employed a refined version of the YOLO face to detect faces from video frames.…”

“…The YOLO detector, one of the most popular state-of-theart face detectors, is used for the face detection process [55,56]. It is characterized by producing fewer false-positive samples compared to other face detectors: BlazeFace, dlib, and MTCNN [21][22][23]. The size of the detected bounding box of the face is increased by 22% relative to its height and width.…”

“…This paper presents a new efficient method, YOLO-Feature extractionmerge hybrid method (YF), which captures the defects in the spatial and temporal domains of video frames and then distinguishes whether a given video is a deepfake or not. The YOLO face detector has been proven its efficiency in detecting the deepfake videos over the other state-of-the-art face detectors since it produces fewer false-positive instances [21][22][23], thus enhancing the performance of the proposed detection method. Therefore, it is used as a face detector extracting the faces from the video frames.…”