DeepfakeUCL: Deepfake Detection via Unsupervised Contrastive Learning

Fung, Sheldon; Lu, Xuequan; Zhang, Chao; Li, Chang-Tsun

doi:10.1109/ijcnn52387.2021.9534089

Cited by 40 publications

(12 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although contrastive learning approaches have been utilized for dense visual representation learning tasks [42], [43], recent studies suggest that removing negative pairs yields better performances [44], [45], [46], [47]. While Random augmentations have been widely used to generate diverse views of the same image [21], [48], [49], this work, inspired by [50], proposes a masked image modeling approach to extract more robust semantic features through a contrastive learning methodology. In contrast to previous work that only focused on mouth regions for lipforensics [51], this work leverages the entire facial region to improve the detection performance.…”

Section: B Self-supervised Contrastive Learningmentioning

confidence: 99%

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

View full text Add to dashboard Cite

The ever-growing threat of deepfakes and large-scale societal implications has propelled the development of deepfake forensics to ascertain the trustworthiness of digital media. A common theme of existing detection methods is using Convolutional Neural Networks (CNNs) as a backbone. While CNNs have demonstrated decent performance on learning local discriminative information, they fail to learn relative spatial features and lose important information due to constrained receptive fields. Motivated by the aforementioned challenges, this work presents DFDT, an end-to-end deepfake detection framework that leverages the unique characteristics of transformer models, for learning hidden traces of perturbations from both local image features and global relationship of pixels at different forgery scales. DFDT is specifically designed for deepfake detection tasks consisting of four main components: patch extraction & embedding, multi-stream transformer block, attention-based patch selection followed by a multi-scale classifier. DFDT’s transformer layer benefits from a re-attention mechanism instead of a traditional multi-head self-attention layer. To evaluate the performance of DFDT, a comprehensive set of experiments are conducted on several deepfake forensics benchmarks. Obtained results demonstrated the surpassing detection rate of DFDT, achieving 99.41%, 99.31%, and 81.35% on FaceForensics++, Celeb-DF (V2), and WildDeepfake, respectively. Moreover, DFDT’s excellent cross-dataset & cross-manipulation generalization provides additional strong evidence on its effectiveness.

show abstract

Section: B Self-supervised Contrastive Learningmentioning

confidence: 99%

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Khormali

Yuan

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Compared with Xception [36], which uses the same backbone network, our method performs better under all quality settings. Our method performs better than DeepfakeUCL [19], which adopts the contrastive learning method. For LQ setting, our method performs inferior to RFAM and F3-Net, similar to Multi-attention.…”

Section: In-dataset Comparison To Other Methodsmentioning

confidence: 89%

“…PixPro [47], DenseCL [44] build pixel-wise contrastive learning framework. There are some recent works [11,19,48] introduce contrastive learning into face forgery detection and get a considerable generalization performance. These methods adopt popular instance discrimination based or pixel wise contrastive learning framework for face forgery detection.…”

Section: Related Workmentioning

confidence: 99%

CORE: Consistent Representation Learning for Face Forgery Detection

Ni¹,

Meng²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Face manipulation techniques develop rapidly and arouse widespread public concerns. Despite that vanilla convolutional neural networks achieve acceptable performance, they suffer from the overfitting issue. To relieve this issue, there is a trend to introduce some erasing-based augmentations. We find that these methods indeed attempt to implicitly induce more consistent representations for different augmentations via assigning the same label for different augmented images. However, due to the lack of explicit regularization, the consistency between different representations is less satisfactory. Therefore, we constrain the consistency of different representations explicitly and propose a simple yet effective framework, COnsistent REpresentation Learning (CORE). Specifically, we first capture the different representations with different augmentations, then regularize the cosine distance of the representations to enhance the consistency. Extensive experiments (in-dataset and cross-dataset) demonstrate that CORE performs favorably against state-of-the-art face forgery detection methods. Our code is available at https://github.com/niyunsheng/CORE.

show abstract

“…The main idea of contrastive learning is to attract similar samples while repel different samples [7,23,27,63]. Recently, some work has used contrastive learning to detect deepfake [19,58,64]. DeepfakeUCL [19] proposes an unsupervised contrastive learning method for deep detection.…”

Section: Relate Workmentioning

confidence: 99%

DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues

Pan,

Yin,

Wei

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

The malicious use and widespread dissemination of deepfake pose a significant crisis of trust. Current deepfake detection models can generally recognize forgery images by training on a large dataset. However, the accuracy of detection models degrades significantly on images generated by new deepfake methods due to the difference in data distribution. To tackle this issue, we present a novel incremental learning framework that improves the generalization of deepfake detection models by continual learning from a small number of new samples. To cope with different data distributions, we propose to learn a domain-invariant representation based on supervised contrastive learning, preventing overfit to the insufficient new data. To mitigate catastrophic forgetting, we regularize our model in both feature-level and label-level based on a multiperspective knowledge distillation approach. Finally, we propose to select both central and hard representative samples to update the replay set, which is beneficial for both domain-invariant representation learning and rehearsal-based knowledge preserving. We conduct extensive experiments on four benchmark datasets, obtaining the new state-of-the-art average forgetting rate of 7.01 and average accuracy of 85.49 on FF++, DFDC-P, DFD, and CDF2. Our code is released at https://github.com/DeepFakeIL/DFIL.

show abstract

DeepfakeUCL: Deepfake Detection via Unsupervised Contrastive Learning

Cited by 40 publications

References 23 publications

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

CORE: Consistent Representation Learning for Face Forgery Detection

DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues

Contact Info

Product

Resources

About