Multi-attentional Deepfake Detection

Abstract

Face forgery by deepfake is widely spread over the internet and has raised severe societal concerns. Recently, how to detect such forgery contents has become a hot research topic and many deepfake detection methods have been proposed. Most of them model deepfake detection as a vanilla binary classification problem, i.e, first use a backbone network to extract a global feature and then feed it into a binary classifier (real/fake). But since the difference between the real and fake images in this task is often subtle and local, we argue this vanilla solution is not optimal. In this paper, we instead formulate deepfake detection as a fine-grained classification problem and propose a new multi-attentional deepfake detection network. Specifically, it consists of three key components: 1) multiple spatial attention heads to make the network attend to different local parts; 2) textural feature enhancement block to zoom in the subtle artifacts in shallow features; 3) aggregate the low-level textural feature and high-level semantic features guided by the attention maps. Moreover, to address the learning difficulty of this network, we further introduce a new regional independence loss and an attention guided data augmentation strategy. Through extensive experiments on different datasets, we demonstrate the superiority of our method over the vanilla binary classifier counterparts, and achieve state-of-the-art performance. The models will be released recently at https://github.com/yoctta/ multiple-attention.

Patch Diffusion: A General Module for Face Manipulation Detection

“…Dang et al (Dang et al 2020) propose an attention mechanism to process the face feature maps, which highlights the informative regions for the improvement of detection ability. Similarly, Zhao et al (Zhao et al 2021a) formulate the face manipulation detection as a fine-grained classification problem, where a multi-attention network is proposed to explore discriminative regions in the face images. Wang et al (Wang et al 2020a) discuss the impact of data augmentation and indicate that using common image processing operations would be able to improve the generality.…”

DeepFake Video Detection through Facial Sparse Optical Flow based Light CNN

“…However, in order to generate the greyscale image which is the same size with the input image, the utilized HRNet [3] requires a significant amount of computing resource. Zhao et al [4] proposed a new multiattentional DeepFake detection network consisting of an Attention Module, a texture enhancement block and a bilinear attention pooling. The architecture of this network is really complicated.…”

FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point Defects

“…Gram-Net [19] and InTeLe [20] explore the texture information of images to improve robustness. A method combining an attention mechanism and texture features was proposed [17]. Instead of designing large, complex neural networks, we efficiently extract features for effective DeepFake detection.…”

“…In addition, a trained detector is likely based on the specific features of one dataset, and cannot extrapolate to other datasets, i.e., it lacks the ability to generalize. To more effectively detect DeepFakes, some recent work has meticulously designed DNNs to combine modules or features with positive detection capabilities, such as an attention mechanism [16,17,18], texture features [19,20], audio and visual modalities [21], and frequency spectrum [22]. Concomitantly, to drive large and complex DNNs with large-scale datasets requires significant computing resources (e.g., GPUs) and training (e.g., parameter adjustment), which decrease efficiency.…”