Two-Stream Neural Networks for Tampered Face Detection

Zhou, Peng; Han, Xintong; Morariu, Vlad I.; Davis, Larry S.

doi:10.1109/cvprw.2017.229

Cited by 504 publications

(272 citation statements)

References 21 publications

Supporting

Mentioning

269

Contrasting

Unclassified

Order By: Relevance

“…Recent work on image forensics utilizes clues such as local noise features [35,26] and Camera Filter Array (CFA) patterns [19] to classify a specific patch or pixel [11] in an image as tampered or not, and localize the tampered regions [19,9,6]. Most of these methods focus on a single tampering technique.…”

Section: Introductionmentioning

confidence: 99%

Learning Rich Features for Image Manipulation Detection

Zhou

Han

Morariu

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

560

440

View full text Add to dashboard Cite

Image manipulation detection is different from traditional semantic object detection because it pays more attention to tampering artifacts than to image content, which suggests that richer features need to be learned. We propose a two-stream Faster R-CNN network and train it endto-end to detect the tampered regions given a manipulated image. One of the two streams is an RGB stream whose purpose is to extract features from the RGB image input to find tampering artifacts like strong contrast difference, unnatural tampered boundaries, and so on. The other is a noise stream that leverages the noise features extracted from a steganalysis rich model filter layer to discover the noise inconsistency between authentic and tampered regions. We then fuse features from the two streams through a bilinear pooling layer to further incorporate spatial co-occurrence of these two modalities. Experiments on four standard image manipulation datasets demonstrate that our two-stream framework outperforms each individual stream, and also achieves state-of-the-art performance compared to alternative methods with robustness to resizing and compression.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Rich Features for Image Manipulation Detection

Zhou

Han

Morariu

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

560

440

View full text Add to dashboard Cite

show abstract

“…While recent state-of-the-art visual forensics techniques demonstrate impressive results for detecting fake visual media [16,53,27,13,22,11,35,67,68,26], they have only focused on semantic, physical, or statistical inconsistency of specific forgery scenarios, e.g., copy-move manipulations [16,26] or face swapping [67]. Forensics on GAN-generated images [44,47,59] shows good accuracy, but each method operates on only one GAN architecture by identifying its unique artifacts and results deteriorate when the GAN architecture is changed.…”

Section: Introductionmentioning

confidence: 99%

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

Davis

Fritz

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

373

267

View full text Add to dashboard Cite

Recent advances in Generative Adversarial Networks (GANs) have shown increasing success in generating photorealistic images. But they also raise challenges to visual forensics and model attribution. We present the first study of learning GAN fingerprints towards image attribution and using them to classify an image as real or GANgenerated. For GAN-generated images, we further identify their sources. Our experiments show that (1) GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution;(2) even minor differences in GAN training can result in different fingerprints, which enables fine-grained model authentication; (3) fingerprints persist across different image frequencies and patches and are not biased by GAN artifacts; (4) fingerprint finetuning is effective in immunizing against five types of adversarial image perturbations; and (5) comparisons also show our learned fingerprints consistently outperform several baselines in a variety of setups 1 .

show abstract

“…Xin et al [13] proposed a new method to detect and identify video forgery based on the inconsistency of the head positions. Peng et al [14] proposed a two-stream network for face tamper detection. GoogleNet is trained to detect tampering artifacts in the face classification stream, and a patch on the basis of the triplet network is trained to leverage features capturing partial noise residuals and camera characteristics as the second stream.…”

Section: Introductionmentioning

confidence: 99%

A Novel Counterfeit Feature Extraction Technique for Exposing Face-Swap Images Based on Deep Learning and Error Level Analysis

Zhang

Zhao

2020

Entropy

View full text Add to dashboard Cite

The quality and efficiency of generating face-swap images have been markedly strengthened by deep learning. For instance, the face-swap manipulations by DeepFake are so real that it is tricky to distinguish authenticity through automatic or manual detection. To augment the efficiency of distinguishing face-swap images generated by DeepFake from real facial ones, a novel counterfeit feature extraction technique was developed based on deep learning and error level analysis (ELA). It is related to entropy and information theory such as cross-entropy loss function in the final softmax layer. The DeepFake algorithm is only able to generate limited resolutions. Therefore, this algorithm results in two different image compression ratios between the fake face area as the foreground and the original area as the background, which would leave distinctive counterfeit traces. Through the ELA method, we can detect whether there are different image compression ratios. Convolution neural network (CNN), one of the representative technologies of deep learning, can extract the counterfeit feature and detect whether images are fake. Experiments show that the training efficiency of the CNN model can be significantly improved by the ELA method. In addition, the proposed technique can accurately extract the counterfeit feature, and therefore achieves outperformance in simplicity and efficiency compared with direct detection methods. Specifically, without loss of accuracy, the amount of computation can be significantly reduced (where the required floating-point computing power is reduced by more than 90%).

show abstract

Two-Stream Neural Networks for Tampered Face Detection

Cited by 504 publications

References 21 publications

Learning Rich Features for Image Manipulation Detection

Learning Rich Features for Image Manipulation Detection

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

A Novel Counterfeit Feature Extraction Technique for Exposing Face-Swap Images Based on Deep Learning and Error Level Analysis

Contact Info

Product

Resources

About