CNN-Based Multi-Modal Camera Model Identification on Video Sequences

Abstract

Identifying the source camera of images and videos has gained significant importance in multimedia forensics. It allows tracing back data to their creator, thus enabling to solve copyright infringement cases and expose the authors of hideous crimes. In this paper, we focus on the problem of camera model identification for video sequences, that is, given a video under analysis, detecting the camera model used for its acquisition. To this purpose, we develop two different CNN-based camera model identification methods, working in a novel multi-modal scenario. Differently from mono-modal methods, which use only the visual or audio information from the investigated video to tackle the identification task, the proposed multi-modal methods jointly exploit audio and visual information. We test our proposed methodologies on the well-known Vision dataset, which collects almost 2000 video sequences belonging to different devices. Experiments are performed, considering native videos directly acquired by their acquisition devices and videos uploaded on social media platforms, such as YouTube and WhatsApp. The achieved results show that the proposed multi-modal approaches significantly outperform their mono-modal counterparts, representing a valuable strategy for the tackled problem and opening future research to even more challenging scenarios.

Source Camera Device Identification from Videos

“…Dal et al [17] proposed a multi-modal ConvNet based approach for camera model identification from video sequences. They combine the visual and the audio signals from a video and show that such an approach would result in a more reliable identification.…”

“…In our work, we focus on SCI based only on visual content, also because in practice audio content can easily be replaced or manipulated. In the visual content based ConvNet, Dal et al [17] pick 50 frames equally spaced in time, and extract 10 patches of size 256 × 256 pixels followed by patch standardization as part of their pre-processing. Furthermore, a pre-trained Effecient-Net [50] was employed for classification.…”

“…The class labels correspond to the sequence of 28 devices listed in Table 1 accuracy of these methods would therefore be limited by the accuracy of determining the presence of video stabilization. The recent work of Cortivo et al [17] included experiments at camera model level with 25 different cameras, trained with only indoor and outdoor samples, and used a data set split of 80:20 for train+val:test. In our experiments, we considered experiments at device level using 28 devices, used all indoor, outdoor and flat scenarios and applied a data set split of 65:35.…”

“…In our experiments, we considered experiments at device level using 28 devices, used all indoor, outdoor and flat scenarios and applied a data set split of 65:35. Cortivo et al [17] trained three different models, one for each compression type using visual content. In our experiments we combine all compression types and train a single model.…”

Image and Video Forensics

“…To this end, Ferrara et al [10] presented a new approach for the performance evaluation of source camera attribution by using likelihood ratio methods obtained from the PRNU similarity scores. Dal Cortivo et al [11] investigated the camera model identification on video proposing a CNN (Convolutional Neural Network) based method jointly exploit audio and visual information. Ferreira A. et al [12] focused their contribution on validating synthetic image detection and source linking methods on a new large scale dataset of printed documents.…”