Learning Patch-Channel Correspondence for Interpretable Face Forgery Detection

Hua, Yingying; Shi, Ruixin; Wang, Pengju; Ge, Shiming

doi:10.1109/tip.2023.3246793

Cited by 21 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ease of access and misuse of deepfake technology has led to the materialization of severe risks, and developing deepfake detec-tion to counteract such threats is all the more pertinent and urgent. Deepfake detection (Ying et al 2023;Ba et al 2023;Hua et al 2023;Wu et al 2023;Pan et al 2023;Shuai et al 2023) faces a significant challenge posed by the sophistication of deepfake technology that can create highly realistic content that is barely distinguishable from real ones.…”

Section: Related Workmentioning

confidence: 99%

Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection

Ba,

Liu,

Liu

et al. 2024

AAAI

View full text Add to dashboard Cite

Deepfake technology has given rise to a spectrum of novel and compelling applications. Unfortunately, the widespread proliferation of high-fidelity fake videos has led to pervasive confusion and deception, shattering our faith that seeing is believing. One aspect that has been overlooked so far is that current deepfake detection approaches may easily fall into the trap of overfitting, focusing only on forgery clues within one or a few local regions. Moreover, existing works heavily rely on neural networks to extract forgery features, lacking theoretical constraints guaranteeing that sufficient forgery clues are extracted and superfluous features are eliminated. These deficiencies culminate in unsatisfactory accuracy and limited generalizability in real-life scenarios. In this paper, we try to tackle these challenges through three designs: (1) We present a novel framework to capture broader forgery clues by extracting multiple non-overlapping local representations and fusing them into a global semantic-rich feature. (2) Based on the information bottleneck theory, we derive Local Information Loss to guarantee the orthogonality of local representations while preserving comprehensive task-relevant information. (3) Further, to fuse the local representations and remove task-irrelevant information, we arrive at a Global Information Loss through the theoretical analysis of mutual information. Empirically, our method achieves state-of-the-art performance on five benchmark datasets. Our code is available at https://github.com/QingyuLiu/Exposing-the-Deception, hoping to inspire researchers.

show abstract

Section: Related Workmentioning

confidence: 99%

Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection

Ba,

Liu,

Liu

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…While this part-based training framework provides interpretability, it struggles to generalize well on visually challenging datasets [6]. Hua et al [179] presented an interpretable face forgery detection model by establishing patchchannel correspondence. The model includes an encoder, a feature-rearranging layer, and a binary classifier.…”

Section: Generic Neural Network Approachmentioning

confidence: 99%

DeepFake on Face and Expression Swap: A Review

Waseem,

Abu Bakar,

Ahmed

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Remarkable advances have been made in deep learning, leading to the emergence of highly realistic AIgenerated videos known as deepfakes. Deepfakes use generative models to manipulate facial features to create modified identities or expressions with impressive realism. These synthetic media creations can deceive, discredit, or blackmail individuals and threaten the integrity of the legal, political, and social systems. Consequently, researchers are actively developing techniques to detect deepfake content to preserve privacy and combat the dissemination of fabricated media. This article presents a comprehensive study examining existing methods of creating deepfake images and videos for face and expression replacement. In addition, it provides an overview of publicly available deepfake datasets for benchmarking, serving as important resources for training and evaluating deepfake detection systems. In addition, the study sheds light on the detection approaches used to identify deepfake face and expression swaps while discussing the challenges and issues involved. However, the focus of this study goes beyond identifying the existing barriers. It goes a step further by outlining future research directions and guiding future scientists to address the concerns that need to be addressed in deepfake detection methods. In this way, this paper aims to facilitate the development of robust and effective deepfake detection solutions for face and expression swaps, thereby contributing to ongoing efforts to protect the authenticity and trustworthiness of visual media.

show abstract

“…The additional generated operation graphs can also serve as better supervision to enhance the performance of face forgery detectors. Hua et al [18] convert the feature reconstruction layer into a deep neural network, and at the same time, classification tasks and correspondence relationships will be optimized. The task is completed through alternative optimization.…”

Section: A Face Forgery Detectionmentioning

confidence: 99%

Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis

Liu

Gao

Peng

et al. 2022

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Deep generator technology can produce high-quality fake videos that are indistinguishable, posing a serious social threat. Traditional forgery detection methods directly centralized training on data and lacked consideration of information sharing in non-public video data scenarios and data privacy. Naturally, the federated learning strategy can be applied for privacy protection, which aggregates model parameters of clients but not original data. However, simple federated learning can't achieve satisfactory performance because of poor generalization capabilities for the real hybrid-domain forgery dataset. To solve the problem, the paper proposes a novel federated face forgery detection learning with personalized representation. The designed Personalized Forgery Representation Learning aims to learn the personalized representation of each client to improve the detection performance of individual client models. In addition, a personalized federated learning training strategy is utilized to update the parameters of the distributed detection model. Here collaborative training is conducted on multiple distributed client devices, and shared representations of these client models are uploaded to the server side for aggregation. Experiments on several public face forgery detection datasets demonstrate the superior performance of the proposed algorithm compared with state-of-the-art methods. The code is available at https://github.com/GANG370/PFR-Forgery.

show abstract

Learning Patch-Channel Correspondence for Interpretable Face Forgery Detection

Cited by 21 publications

References 64 publications

Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection

Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection

DeepFake on Face and Expression Swap: A Review

Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis

Contact Info

Product

Resources

About