Design Principles of Convolutional Neural Networks for Multimedia Forensics

Abstract

Convolutional neural networks (CNNs) have received significant attention due to their ability to adaptively learn classification features directly from data. While CNNs have helped cause dramatic advances in fields such as object and speech recognition, multimedia forensics is fundamentally different problem compared to other deep learning applications. Little work exists to guide the design of CNN architectures for forensic tasks. Furthermore, it is still unclear which forensic tasks can be performed using CNNs. In this work, we investigate the design of CNNs for multiple multimedia forensic applications. We show that CNNs are capable of performing image manipulation detection as well as camera model identification. Through a series of experiments, we systematically examine the influence of several important CNN design choices for forensic applications, such as the use of a constrained convolutional layer or fixed high-pass filter at the beginning of the CNN, the use of nonlinearity after the first layer, the choice of activation and pooling functions, etc. We show that different CNN design choices should be made for different forensic applications and identify design choices to maximize the performance of CNNs for manipulation detection and camera model identification.

Forensic Similarity for Digital Images

“…Furthermore, it has been shown that deep features from a CNN trained for camera model identification transfer very well to other forensic tasks such manipulation detection [23], suggesting that deep features related to digital forensics are general to a variety of forensics tasks. 1) Architecture: To build a forensic feature extractor, we adapt the MISLnet CNN architecture developed in [7], which has been utilized in a number of works that target different digital image forensics tasks including manipulation detection [6], [7], [30] and camera model identification [23], [30]. Briefly, this CNN consists of 5 convolutional blocks, labeled 'conv1' through 'conv5' in Fig.…”

MultiMedia Modeling

“…The robustness against post-processing is not evaluated and it proposes in the future to detect image forgeries. The work in [4] examines the influence of several important CNN design choices for forensic applications, such as the use of a constrained convolutional layer or fixed high-pass filter at the beginning of the CNN. In [7,8], two techniques are combined for image tampering detection and localisation, leveraging characteristic footprints left on images by different camera models.…”

“…Patches extraction: As state-of-the-art methods [4,8] for camera model classification gives promising results with small image patches, we also divide our image in small patches (64×64 pixels) as the second step of our robust framework for camera model identification. Indeed, the use of small image patches instead of full-resolution images better characterizes camera models in a reduced-size space.…”

Detection and Localization of Image Forgeries Using Resampling Features and Deep Learning

“…Recent efforts in detecting manipulations exploit deep learning based model in [6,4,36]. These include detection of generic manipulations [6,4], resampling [5], splicing [36] and bootleg [10]. The authors tested existing CNN network for steganalysis [35].…”