Exploiting Spatial Structure for Localizing Manipulated Image Regions

Abstract

The advent of high-tech journaling tools facilitates an image to be manipulated in a way that can easily evade state-of-the-art image tampering detection approaches. The recent success of the deep learning approaches in different recognition tasks inspires us to develop a high confidence detection framework which can localize manipulated regions in an image. Unlike semantic object segmentation where all meaningful regions (objects) are segmented, the localization of image manipulation focuses only the possible tampered region which makes the problem even more challenging. In order to formulate the framework, we employ a hybrid CNN-LSTM model to capture discriminative features between manipulated and non-manipulated regions. One of the key properties of manipulated regions is that they exhibit discriminative features in boundaries shared with neighboring non-manipulated pixels. Our motivation is to learn the boundary discrepancy, i.e., the spatial structure, between manipulated and non-manipulated regions with the combination of LSTM and convolution layers. We perform end-to-end training of the network to learn the parameters through back-propagation given groundtruth mask information. The overall framework is capable of detecting different types of image manipulations, including copy-move, removal and splicing. Our model shows promising results in localizing manipulated regions, which is demonstrated through rigorous experimentation on three diverse datasets.

Hybrid LSTM and Encoder–Decoder Architecture for Detection of Image Forgeries

“…It will also help train deeper networks for image tamper classification or localization tasks. This work builds upon our earlier paper [7], but with significant differences. First, the method presented in the paper [7] exploits low level features such as tampered edges, as evidence of tampering, which cannot always detect the entire tampered regions.…”

“…Finally, finetuning this 'LSTM-EnDec-Base' model provides a boost in performance for labeling tamper class at pixel level. From the table, we can see that proposed model 'LSTM-EnDec' outperforms 'LSTM-EnDec-Base' model [52] 0.545 --ADJPEG [13] 0.5891 --NADJPEG [13] 0.6567 --PatchMatch [24] 0.6513 --Error level analysis [57] 0.4288 --Block Features [49] 0.4785 --Noise Inconsistencies [60] 0.4874 --J-Conv-LSTM-Conv [7] 0.7641 0.7238 0.6137 LSTM-EnDec 0.7936 0.7577 0.7124 by 3.44%, and 2.95% on NIST'16 [2], IEEE Forensics Challenge [1] datasets respectively.…”

“…In [7], [17], LSTM network is utilized in order to learn the transition (change) between manipulated vs non-manipulated blocks by feeding the blocks into an LSTM network. In [17], the authors propose a patch classification framework where frequency domain features are extracted from 8 × 8 block before LSTM network.…”

“…FCN : Fully convolutional network as presented in [55]. J-Conv-LSTM-Conv: This method utilizes LSTM network and convolutional layers for segmentation as in [7].…”

“…Moreover, it seems unrealistic to assume that the type of manipulation will be known beforehand. Our recent paper [7], upon which this particular work builds, presents a general detection architecture for different content-changing…”