Attentive Semantic Exploring for Manipulated Face Detection

Abstract

Face manipulation methods develop rapidly in recent years, whose potential risk to society accounts for the emerging of researches on detection methods. However, due to the diversity of manipulation methods and the high quality of fake images, detection methods suffer from a lack of generalization ability. To solve the problem, we find that segmenting images into semantic fragments could be effective, as discriminative defects and distortions are closely related to such fragments. Besides, to highlight discriminative regions in fragments and to measure contribution to the final prediction of each fragment is efficient for the improvement of generalization ability. Therefore, we propose a novel manipulated face detection method based on Multilevel Facial Semantic Segmentation and Cascade Attention Mechanism. To evaluate our method, we reconstruct two datasets: GGFI and FFMI, and also collect two open-source datasets. Experiments on four datasets verify the advantages of our approach against other state-of-the-arts, especially its generalization ability.

An Eyes-Based Siamese Neural Network for the Detection of GAN-Generated Face Images

“…Future work will try to further improve the generalization and robustness capability of the detector by performing fusion with other detectors resorting to semantic analysis, that is, looking at other semantic facial attributes [e.g., the mouth (Suwajanakorn et al, 2017;Haliassos et al, 2021), or the nose (Chen and Yang, 2021)]. The existence of inconsistencies in symmetries that might come from other facial attributes, e.g., eyebrows or mouths shapes, is also worth investigation.…”

“…In Liu et al (2020), the authors presented a new CNN-based detector, called Gram-Net, that leverages global image texture representations to improve the generalization and the robustness of GAN image detection. An approach that relies on semantic segmentation and perform detection based on multiple semantic fragments getting remarkable generalization capability has been recently presented in Chen and Yang (2021). A different solution to improve generalization is proposed in Xuan et al (2019): the idea is to carry out augmentation by Gaussian blurring so as to force the discriminator to learn more general features.…”

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

“…Multi-task learning could also be used for classifying and locating the manipulated facial images (Nguyen et al, 2019a). Formulating fake forensics as a segmentation task to localize the manipulated region in synthesized faces is another interesting idea in fighting DeepFakes (Li et al, 2019a;Chen and Yang, 2020). Combing deep learning and co-occurrence matrices could also be used for the detection, attribution, and localization of GAN images (Goebel et al, 2020).…”

Automated Deepfake Detection

“…• LAE [39] utilizes an encoder-decoder network for simultaneously predicting real and fake labels as well as locating the manipulated regions. We do not compare our work with Face X-ray [41] since it needs external data collection and annotation for training their network; we do compare our work with [22], [60] since they conduct additional inference task other than real/fake prediction and manipulation mask segmentation, and therefore bring additional computation cost and increase network complexity. For example, [22] brings a facial expression recognition module into their architecture to recognize facial expressions, [60] conducts a multilevel facial semantic segmentation in their architecture.…”