FakeTalkerDetect: Effective and Practical Realistic Neural Talking Head Detection with a Highly Unbalanced Dataset

Jeon, Hyeonseong; Bang, Youngoh; Woo, Simon S.

doi:10.1109/iccvw.2019.00163

Cited by 22 publications

(11 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After that, we apply the SVM as our learning model. To demonstrate our approach's effectiveness, we compare our method with FakeTalkerDetect model [13], which deployed a pre-trained AlexNet and Siamese network trained on RGB images. The results are presented in Table 3.…”

Section: Resultsmentioning

confidence: 99%

“…Although it would be challenging for human eyes to distinguish between real and GAN-generated fake images, we believe that their frequency spectra differences can be possibly exposed, when we stack the three channels' spectra of real vs. fake. Figure 1 presents our example of images' spectra from VoxCeleb2 dataset [12] and Fake Head Talker dataset [13]. In particular, in the real images, we empirically find that the spectra of three color channels are mostly concurrent when stacking together, whereas they become noisy in the fake images, as shown in Fig.…”

Section: Descriptive Features Extractionmentioning

confidence: 86%

See 1 more Smart Citation

Exploring the Asynchronous of the Frequency Spectra of GAN-generated Facial Images

Le¹,

Woo²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The rapid progression of Generative Adversarial Networks (GANs) has raised a concern of their misuses for malicious purposes, especially in creating fake face images. Although many proposed methods succeed in detecting GAN-based synthetic images, they are still limited by the need for large quantities of the training fake image dataset, and challenges for the detector's generalizability to unknown facial images. In this paper, we propose a new approach that explores the asynchronous frequency spectra of color channels, which is simple but effective for training both unsupervised and supervised learning models to distinguish GAN-based synthetic images. We further investigate the transferability of a training model that learns from our suggested features in one source domain and validates on another target domains with prior knowledge of the features' distribution. Our experimental results show that the discrepancy of spectra in the frequency domain is a practical artifact to effectively detect various types of GAN-based generated images.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Descriptive Features Extractionmentioning

confidence: 86%

Exploring the Asynchronous of the Frequency Spectra of GAN-generated Facial Images

Le¹,

Woo²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…While unimodal DeepFake Detection methods (discussed in Section 2.2) have focused only on the facial features of the subject, there has not been much focus on using the multiple modalities that are part of the same video. Jeo and Bang et al [30] propose FakeTalkerDetect, which is a Siamese-based network to detect the fake videos generated from the neural talking head models. They perform a classification based on distance.…”

Section: Multimodal Deepfake Detection Methodsmentioning

confidence: 99%

Emotions Don't Lie

Mittal

Bhattacharya

Chandra

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

179

View full text Add to dashboard Cite

We present a learning-based method for detecting real and fake deepfake multimedia content. To maximize information for learning, we extract and analyze the similarity between the two audio and visual modalities from within the same video. Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake". We propose a deep learning network, inspired by the Siamese network architecture and the triplet loss. To validate our model, we report the AUC metric on two large-scale deepfake detection datasets, DeepFake-TIMIT Dataset and DFDC. We compare our approach with several SOTA deepfake detection methods and report per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets, respectively. To the best of our knowledge, ours is the first approach that simultaneously exploits audio and video modalities and also perceived emotions from the two modalities for deepfake detection. CCS CONCEPTS • Computing methodologies → Visual content-based indexing and retrieval; Neural networks.

show abstract

“…We used DeepFakes and StarGAN, each from the FaceForensics++ and GAN image dataset, to compare the results. We compare our DA-FDFtNet with the original FDFtNet [14], which does not include the channel attention module to capture the channel features. We used the same hyperparameter settings from the previous experiment for both FDFtNet and DA-FDFtNet.…”

Section: Ablation Studymentioning

confidence: 99%

“…Therefore, previous metadata-based detection approaches are not practical and useful against recent fake images generated from GANs. In order to address these issues, Convolutional Neural Networks (CNNs)-based binary classifiers such as ShallowNet [13], FakeTalkerDetect [14], FaceForen-sics++ [15], and Face X-ray [16] are developed, training with a large number of real vs. forged images. Furthermore, other researchers [17,18,19] have shown that the detection performance can be improved by analyzing artifacts and patterns in underlying GAN-images.…”

Section: Introductionmentioning

confidence: 99%

DA-FDFtNet: Dual Attention Fake Detection Fine-tuning Network to Detect Various AI-Generated Fake Images

Bang¹,

Woo²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the advancement of Generative Adversarial Networks (GAN), Autoencoders, and other AI technologies, it has been much easier to create fake images such as "Deepfakes." More recent research has introduced few-shot learning, which uses a small amount of training data to produce fake images and videos more effectively. Therefore, the ease of generating manipulated images and the difficulty of distinguishing those images can cause a serious threat to our society, such as propagating fake information. However, detecting realistic fake images generated by the latest AI technology is challenging due to the reasons mentioned above. In this work, we propose Dual Attention Fake Detection Fine-tuning Network (DA-FDFtNet) to detect the manipulated fake face images from the real face data. Our DA-FDFtNet integrates the pre-trained model with Fine-Tune Transformer, MBblockV3, and a channel attention module to improve the performance and robustness across different types of fake images. In particular, Fine-Tune Transformer consists of multiple numbers of an image-based self-attention module and a down-sampling layer. The channel attention module is also connected with the pre-trained model to capture the fake images feature space. We experiment with our DA-FDFtNet with the FaceForensics++ dataset and various GAN-generated datasets, and we show that our approach outperforms the previous baseline models.

show abstract

FakeTalkerDetect: Effective and Practical Realistic Neural Talking Head Detection with a Highly Unbalanced Dataset

Cited by 22 publications

References 2 publications

Exploring the Asynchronous of the Frequency Spectra of GAN-generated Facial Images

Exploring the Asynchronous of the Frequency Spectra of GAN-generated Facial Images

Emotions Don't Lie

DA-FDFtNet: Dual Attention Fake Detection Fine-tuning Network to Detect Various AI-Generated Fake Images

Contact Info

Product

Resources

About