Fighting Fake News: Image Splice Detection via Learned Self-Consistency

Abstract

Fig. 1: Our algorithm learns to detect and localize image manipulations (splices), despite being trained only on unmanipulated images. The two input images above might look plausible, but our model correctly determined that they have been manipulated because they lack self-consistency: the visual information within the predicted splice region was found to be inconsistent with the rest of the image. IMAGE CREDITS: automatically created splice from Hays and Efros [1] (top), manual splice from Reddit user /u/Name-Albert Einstein (bottom).Abstract. Advances in photo editing and manipulation tools have made it significantly easier to create fake imagery. Learning to detect such manipulations, however, remains a challenging problem due to the lack of sufficient amounts of manipulated training data. In this paper, we propose a learning algorithm for detecting visual image manipulations that is trained only using a large dataset of real photographs. The algorithm uses the automatically recorded photo EXIF metadata as supervisory signal for training a model to determine whether an image is self-consistent -that is, whether its content could have been produced by a single imaging pipeline. We apply this self-consistency model to the task of detecting and localizing image splices. The proposed method obtains state-ofthe-art performance on several image forensics benchmarks, despite never seeing any manipulated images at training. That said, it is merely a step in the long quest for a truly general purpose visual forensics tool. Fig. 2: Anatomy of a splice: One of the most common ways of creative fake images is splicing together content from two different real source images. The insight explored in this paper is that patches from a spliced image are typically produced by different imaging pipelines, as indicated by the EXIF meta-data of the two source images. The problem is that in practice, we never have access to these source images at test time. 1

Detecting Photoshopped Faces by Scripting Photoshop

“…(1) FaceForensics++ [31]: A network trained on face swapping and reenactment data; we use the Xception [12] model trained on raw video frames. (2) Selfconsistency [15]: A network trained to spot low-level inconsistencies within an image.…”

Hue Modification Localization By Pair Matching

“…We propose to verify the inconsistency of P i and P j by means of a Siamese neural network [9]. Siamese neural networks have been recently exploited for applications in multimedia forensics [10]- [12]. This network architecture consists of two identical sub-networks f θ , followed by a non-linear classifier g γ that outputs an inconsistency score z ij whose standard logistic activation is defined as:…”

“…It is typical to assume that the forged region is relatively small compared to the background, thus majority of I k (k refers to patches on the pristine region) exposes inconsistencies with the forged region, while remaining maps expose inconsistencies with the pristine region, as shown in Figure 3. In order to fuse inconsistency maps of majority patches belonging to the pristine region to obtain a unique map I ∈ R N H ×N W , we follow the approach in [12], computingĪ by mean shift algorithm [17], which iteratively finds mean of majority (mode). Eventually,Ī is a subsampled heatmap which potentially highlights malicious region.…”

Fact-Checking Meets Fauxtography: Verifying Claims About Images

“…A common manipulation is splicing, i.e., cropping and stitching together parts of the same image or multiple different images. We explored an approach that looks for the lack of self-consistency in images and outputs clusters of the predicted image parts using two algorithms: MeanShift and DBSCAN (Huh et al, 2018). An illustration on how it works is shown in Figure 7.…”