ADT: Anti-Deepfake Transformer

Wang, Ping; Liu, Kunlin; Zhou, Wenbo; Zhou, Hang; Liu, Honggu; Zhang, Weimin; Yu, Nenghai

doi:10.1109/icassp43922.2022.9746888

Cited by 12 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In light of recent advancements in deepfake generation techniques that have reduced the visual anomalies, the detection methods have shifted focus towards more sophisticated approaches such as attention mechanisms [29], [37], [16], [8], [38] and vision Transformers [17], [24], [39]. For example, Wang et al [24] proposed a multi-regional attention mechanism to enhance deepfake detection performance.…”

Section: A Deepfake Detectionmentioning

confidence: 99%

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

View full text Add to dashboard Cite

The ever-growing threat of deepfakes and large-scale societal implications has propelled the development of deepfake forensics to ascertain the trustworthiness of digital media. A common theme of existing detection methods is using Convolutional Neural Networks (CNNs) as a backbone. While CNNs have demonstrated decent performance on learning local discriminative information, they fail to learn relative spatial features and lose important information due to constrained receptive fields. Motivated by the aforementioned challenges, this work presents DFDT, an end-to-end deepfake detection framework that leverages the unique characteristics of transformer models, for learning hidden traces of perturbations from both local image features and global relationship of pixels at different forgery scales. DFDT is specifically designed for deepfake detection tasks consisting of four main components: patch extraction & embedding, multi-stream transformer block, attention-based patch selection followed by a multi-scale classifier. DFDT’s transformer layer benefits from a re-attention mechanism instead of a traditional multi-head self-attention layer. To evaluate the performance of DFDT, a comprehensive set of experiments are conducted on several deepfake forensics benchmarks. Obtained results demonstrated the surpassing detection rate of DFDT, achieving 99.41%, 99.31%, and 81.35% on FaceForensics++, Celeb-DF (V2), and WildDeepfake, respectively. Moreover, DFDT’s excellent cross-dataset & cross-manipulation generalization provides additional strong evidence on its effectiveness.

show abstract

Section: A Deepfake Detectionmentioning

confidence: 99%

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Lin et al [22] used convolution layers at different levels to capture high-level semantic features of face images and performed feature fusion, refinement and reorganization to achieve better generalization performance than the baseline method. Wang et al [23] proposed a transformation-based framework for modeling and analyzing global and local information of facial images. The method relies less on local texture features in the training data, which increases its generality.…”

Section: B Generalized Forgery Detectionmentioning

confidence: 99%

Hierarchical Forgery Classifier on Multi-Modality Face Forgery Clues

Liu

Zeyang

Peng

et al. 2024

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Due to the successful development of deep image generation technology, visual data forgery detection would play a more important role in social and economic security. Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain.In this paper, we propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF). Most forgery technologies always bring in high-frequency aware cues, which make it easy to distinguish source authenticity but difficult to generalize to unseen artifact types. The masked frequency forgery representation module is designed to explore robust forgery cues by randomly discarding high-frequency information. In addition, we find that the forgery attention map inconsistency through the detection network could affect the generalizability. Thus, the forgery attention consistency is introduced to force detectors to focus on similar attention regions for better generalization ability. Experiment results on several public face forgery datasets (Face-Forensic++, DFD, Celeb-DF, and WDF datasets) demonstrate the superior performance of the proposed method compared with the state-of-the-art methods. The source code and models are publicly available at https://github.com/chenboluo/ACMF.

show abstract

“…In light of recent advancements in deepfake generation techniques that have reduced the visual anomalies, the detection methods have shifted focus towards more sophisticated approaches such as attention mechanisms [8], [16], [29], [37], [38] and vision Transformers [17], [24], [39]. For example, Wang et al [24] proposed a multi-regional attention mechanism to enhance deepfake detection performance.…”

Section: Related Work a Deepfake Detectionmentioning

confidence: 99%

Self-Supervised Graph Transformer for Deepfake Detection

Khormali,

Yuan

2024

IEEE Access

View full text Add to dashboard Cite

Deepfake detection methods have shown promising results in recognizing forgeries within a given dataset, where training and testing take place on the in-distribution dataset. However, their performance deteriorates significantly when presented with unseen samples. As a result, a reliable deepfake detection system must remain impartial to forgery types, appearance, and quality for guaranteed generalizable detection performance. Despite various attempts to enhance cross-dataset generalization, the problem remains challenging, particularly when testing against common post-processing perturbations, such as video compression or blur. Hence, this study introduces a deepfake detection framework, leveraging a self-supervised pre-training model that delivers exceptional generalization ability, withstanding common corruptions and enabling feature explainability. The framework comprises three key components: a feature extractor based on vision Transformer architecture that is pre-trained via self-supervised contrastive learning methodology, a graph convolution network coupled with a Transformer discriminator, and a graph Transformer relevancy map that provides a better understanding of manipulated regions and further explains the model's decision. To assess the effectiveness of the proposed framework, several challenging experiments are conducted, including in-data distribution performance, cross-dataset & cross-manipulation generalization, and robustness against common post-production perturbations. The results achieved demonstrate the remarkable effectiveness of the proposed deepfake detection framework, surpassing the current state-of-theart approaches.

show abstract

ADT: Anti-Deepfake Transformer

Cited by 12 publications

References 19 publications

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Hierarchical Forgery Classifier on Multi-Modality Face Forgery Clues

Self-Supervised Graph Transformer for Deepfake Detection

Contact Info

Product

Resources

About