Recasting Residual-based Local Descriptors as Convolutional Neural Networks

Cozzolino, Davide; Poggi, Gianfranco; Verdoliva, Luisa

doi:10.1145/3082031.3083247

Cited by 322 publications

(168 citation statements)

References 28 publications

Supporting

Mentioning

167

Contrasting

Unclassified

Order By: Relevance

“…In particular, for the network proposed in Cozzolino at al. [17] the used learning-rate is 10 −5 with batch-size 16. For the proposal of Bayar and Stamm [10], we use a learning-rate equal to 10 −5 with a batch-size of 64.…”

Section: D2 Classification Methodsmentioning

confidence: 99%

FaceForensics++: Learning to Detect Manipulated Facial Images

Rössler

Cozzolino

Verdoliva

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

1,814

1,639

View full text Add to dashboard Cite

Figure 1: FaceForensics++ is a dataset of facial forgeries that enables researchers to train deep-learning-based approaches in a supervised fashion. The dataset contains manipulations created with four state-of-the-art methods, namely, Face2Face, FaceSwap, DeepFakes, and NeuralTextures. AbstractThe rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-ofthe-art image manipulations, and how difficult it is to detect them, either automatically or by humans.To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection 1 . In particular, the benchmark is based on Deep-Fakes [1], Face2Face [59], FaceSwap [2] and NeuralTextures [57] as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available 2 and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domainspecific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.

show abstract

Section: D2 Classification Methodsmentioning

confidence: 99%

FaceForensics++: Learning to Detect Manipulated Facial Images

Rössler

Cozzolino

Verdoliva

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

1,814

1,639

View full text Add to dashboard Cite

show abstract

“…While recent state-of-the-art visual forensics techniques demonstrate impressive results for detecting fake visual media [16,53,27,13,22,11,35,67,68,26], they have only focused on semantic, physical, or statistical inconsistency of specific forgery scenarios, e.g., copy-move manipulations [16,26] or face swapping [67]. Forensics on GAN-generated images [44,47,59] shows good accuracy, but each method operates on only one GAN architecture by identifying its unique artifacts and results deteriorate when the GAN architecture is changed.…”

Section: Introductionmentioning

confidence: 99%

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

Davis

Fritz

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

377

267

View full text Add to dashboard Cite

Recent advances in Generative Adversarial Networks (GANs) have shown increasing success in generating photorealistic images. But they also raise challenges to visual forensics and model attribution. We present the first study of learning GAN fingerprints towards image attribution and using them to classify an image as real or GANgenerated. For GAN-generated images, we further identify their sources. Our experiments show that (1) GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution;(2) even minor differences in GAN training can result in different fingerprints, which enables fine-grained model authentication; (3) fingerprints persist across different image frequencies and patches and are not biased by GAN artifacts; (4) fingerprint finetuning is effective in immunizing against five types of adversarial image perturbations; and (5) comparisons also show our learned fingerprints consistently outperform several baselines in a variety of setups 1 .

show abstract

“…To detect GAN images, we compute co-occurrence matrices on the RGB channels of an image. Co-occurrence matrices have been previously used in steganalysis to identify images that have data hidden in them [5,6,7,8] and in image forensics to detect or localize tampered images [50,9]. In prior works, cooccurrence matrices are usually computed on image residuals by passing the image through many filters and then obtaining the difference.…”

Section: Methodsmentioning

confidence: 99%

“…Here the authors compare various existing methods to identify cycleGAN images from normal ones. The top results they obtained using a combination of residual features [50,9] and deep learning [51]. Similar to [36], the authors in [38] compute the residuals of high pass filtered images and then extract co-occurrence matrices on these residuals, which are then concatenated to form a feature vector that can distinguish real from fake GAN images.…”

Section: Related Workmentioning

confidence: 99%

“…Traditionally, this method uses hand crafted features computed on the co-occurrence matrix and a machine learning classifier such as support vector machines determines if a message is hidden in the image [5,6]. Other techniques involve calculating image residuals or passing the image through different filters before computing the co-occurrence matrix [7,8,9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting GAN generated Fake Images using Co-occurrence Matrices

Nataraj¹,

Mohammed²,

Manjunath³

et al. 2019

204

169

View full text Add to dashboard Cite

The advent of Generative Adversarial Networks (GANs) has brought about completely novel ways of transforming and manipulating pixels in digital images. GAN based techniques such as Image-to-Image translations, DeepFakes, and other automated methods have become increasingly popular in creating fake images. In this paper, we propose a novel approach to detect GAN generated fake images using a combination of co-occurrence matrices and deep learning. We extract co-occurrence matrices on three color channels in the pixel domain and train a model using a deep convolutional neural network (CNN) framework. Experimental results on two diverse and challenging GAN datasets comprising more than 56,000 images based on unpaired image-to-image translations (cycleGAN [1]) and facial attributes/expressions (StarGAN [2]) show that our approach is promising and achieves more than 99% classification accuracy in both datasets. Further, our approach also generalizes well and achieves good results when trained on one dataset and tested on the other.

show abstract

Recasting Residual-based Local Descriptors as Convolutional Neural Networks

Cited by 322 publications

References 28 publications

FaceForensics++: Learning to Detect Manipulated Facial Images

FaceForensics++: Learning to Detect Manipulated Facial Images

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

Detecting GAN generated Fake Images using Co-occurrence Matrices

Contact Info

Product

Resources

About