Generate, Segment, and Refine: Towards Generic Manipulation Segmentation

Abstract

Detecting manipulated images has become a significant emerging challenge. The advent of image sharing platforms and the easy availability of advanced photo editing software have resulted in a large quantities of manipulated images being shared on the internet. While the intent behind such manipulations varies widely, concerns on the spread of false news and misinformation is growing. Current state of the art methods for detecting these manipulated images suffers from the lack of training data due to the laborious labeling process. We address this problem in this paper, for which we introduce a manipulated image generation process that creates true positives using currently available datasets. Drawing from traditional work on image blending, we propose a novel generator for creating such examples. In addition, we also propose to further create examples that force the algorithm to focus on boundary artifacts during training. Strong experimental results validate our proposal.

Convolutional neural network initialization approaches for image manipulation detection

“…First, when we know what original images should be and are very strict about the manipulations (i.e., any image processing operation is considered suspicious and not tolerable, for example in law enforcement), our approaches could be directly applied in a sliding window manner to locate the suspicious patches. Second, even if routine manipulations can be tolerated, it is in general believed that there may still exist special forgery traces within the fake region and near the fake region boundary, in for example splicing and copy-move forgeries [41,42]. Therefore, still with the application of our approaches on small local patches in a sliding window manner, additionally combined with an efficient clustering algorithm on the extracted features of a manipulation detection CNN, we may be able to locate the fake regions in image forgeries.…”

A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics

“…The proposed end-to-end network used the Noiseprint [ 84 ] as features extracted from the image input. Meanwhile, in [ 91 ] a GAN was proposed to generate falsified images avoiding the burdensome task of creating and labeling image forgery examples in a conventional way. With this big number of synthetic examples, the proposed algorithm was able to segment and refine the focus on boundary artifacts around falsified regions during the training process.…”

Image Manipulation Detection by Multi-View Multi-Scale Supervision

“…Unsurprisingly, the state-of-the-arts are deep learning based [1,15,23,28,29,31], specifically focusing on pixellevel manipulation detection [23,28,31]. With only two classes (manipulated versus authentic) in consideration, the task appears to be a simplified case of image semantic segmentation.…”

“…However, an off-the-shelf semantic segmentation network is suboptimal for the task, as it is designed to capture semantic information, making the network datasetdependent and do not generalize. Prior research [31] re-Figure 1. Image manipulation detection by the state-of-thearts.…”

“…Depending on at what stage the suppression occurs, we categorize existing methods into two groups, i.e. noise-view methods [15,17,28,29,32] and edge-supervised methods [23,31]. Given the hypothesis that novel elements introduced by slicing and/or inpainting differ from the authentic part in terms of their noise distributions, the first group of methods aim to exploit such discrepancy.…”