Deep Convolutional Pooling Transformer for Deepfake Detection

Abstract

Recently, Deepfake has drawn considerable public attention due to security and privacy concerns in social media digital forensics. As the wildly spreading Deepfake videos on the Internet become more realistic, traditional detection techniques have failed in distinguishing between real and fake. Most existing deep learning methods mainly focus on local features and relations within the face image using convolutional neural networks as a backbone. However, local features and relations are insufficient for model training to learn enough general information for Deepfake detection. Therefore, the existing Deepfake detection methods have reached a bottleneck to further improve the detection performance. To address this issue, we propose a deep convolutional Transformer to incorporate the decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. Moreover, we employ the barely discussed image keyframes in model training for performance improvement and visualize the feature quantity gap between the key and normal image frames caused by video compression. We finally illustrate the transferability with extensive experiments on several Deepfake benchmark datasets. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.

A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods

“…Wang et al [45] proposed a deep convolutional transformer to incorporate the decisive image features both locally and globally. The authors applied convolutional pooling and reattention to enrich the extracted features and enhance efficacy.…”

“…Another main challenge is raised due to the "unseen class of some facial datasets" in the testing dataset with respect to the training dataset. Augmenting training datasets with diverse samples, employing transfer learning from pre-trained models, and integrating attention mechanisms can enhance generalization capabilities [45]. Future research topics can include:…”

A survey on deepfake video detection datasets

“…In Table 3, we show a collection of articles (Reference) and their average scores according to the datasets. Faceforensics++ [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38] 94.2 [39], [40], [41], [42], [43] 93. 55 3 DeepFakeDetection [44] 90.80 --4 UADFV [45], [46], [47] 93.4 [48], [49] 98.7 5…”

Hybrid network of convolutional neural network and transformer for deepfake geographic image detection

“…For instance, GANs can be employed to create false statements by politicians and spread misleading military information about adversaries. Therefore, the issue of deepfake detection has drawn the attention of researchers, and they have proposed various methods [7][8][9][10] to counter these disturbing misuses.…”