SSTNet: Detecting Manipulated Faces Through Spatial, Steganalysis and Temporal Features

Wu, Xi; Xie, Zhipeng; Gao, Yutao; Xiao, Yu

doi:10.1109/icassp40776.2020.9053969

Cited by 72 publications

(43 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Forgery Detection Methods Combining Spatial and Frequencymentioning

confidence: 99%

“…To extract features comprehensively and complementary, considering both spatial and frequency domain information gradually becomes the research mainstream [42,43,44,45,28]. SSTNet [42] detected manipulated face images by extracting spatial, steganalysis, and temporal features with modified Xception and LSTM. Two-branch RN [43] combined information from both spatial and frequency domain, and a Laplacian of Gaussian (LOG) was used to enhance multiband frequencies.…”

Section: Forgery Detection Methods Combining Spatial and Frequencymentioning

confidence: 99%

See 1 more Smart Citation

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

Wang¹,

Jiang²,

Jin³

et al. 2021

Preprint

View full text Add to dashboard Cite

As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has become a recent research hotspot. Most existing detection methods train a binary classifier under global supervision to judge real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capture subtle and local forgery artifacts, especially in high compression settings and cross-dataset scenarios. To address such limitations, we propose a novel framework named Multi-modal Contrastive Classification by Locally Correlated Representations (MC-LCR), for effective face forgery detection. Instead of specific appearance features, our MC-LCR aims to amplify implicit local discrepancies between authentic and forged faces from both spatial and frequency domains. Specifically, we design the shallow style representation block that measures the pairwise correlation of shallow feature maps, which encodes local style information to extract more discriminative features in the spatial domain. Moreover, we make a key observation that subtle forgery artifacts can be further exposed in the patch-wise phase and amplitude spectrum and exhibit different clues. According to the complementarity of amplitude and phase information, we develop a patch-wise amplitude and phase dual attention module to capture locally correlated inconsistencies with each other in the frequency domain. Besides the above two modules, we further introduce the collaboration of supervised contrastive loss with cross-entropy loss. It helps the network learn more discriminative and generalized representations. Through extensive experiments and comprehensive studies, we achieve state-of-the-art performance and demonstrate the robustness and generalization of our method.

show abstract

Section: Forgery Detection Methods Combining Spatial and Frequencymentioning

confidence: 99%

Section: Forgery Detection Methods Combining Spatial and Frequencymentioning

confidence: 99%

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

Wang¹,

Jiang²,

Jin³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The performance of Lipschitz regularization in the white box scenario only improves by 2.2 percent, and the DIP method shows higher performance than that of Lipschitz regularization; however, the detection process is highly timeconsuming even after a high-performance configuration. Wu et al [77] introduced an SSTNet method that combines spatial, steganalysis and feature extracted procedures to detect DeepFakes. Basically, XceptionNet is used to monitor the spatial features and statistical information of the image.…”

Section: ) Dnn-based Techniques For Deepfakesmentioning

confidence: 99%

DeepFake Detection for Human Face Images and Videos: A Survey

et al. 2022

View full text Add to dashboard Cite

Techniques for creating and manipulating multimedia information have progressed to the point where they can now ensure a high degree of realism. DeepFake is a generative deep learning algorithm that creates or modifies face features in a superrealistic form, in which it is difficult to distinguish between real and fake features. This technology has greatly advanced and promotes a wide range of applications in TV channels, video game industries, and cinema, such as improving visual effects in movies, as well as a variety of criminal activities, such as misinformation generation by mimicking famous people. To identify and classify DeepFakes, research in DeepFake detection using deep neural networks (DNNs) has attracted increased interest. Basically, DeepFake is the regenerated media that is obtained by injecting or replacing some information within the DNN model. In this survey, we will summarize the DeepFake detection methods in face images and videos on the basis of their results, performance, methodology used and detection type. We will review the existing types of DeepFake creation techniques and sort them into five major categories. Generally, DeepFake models are trained on DeepFake datasets and tested with experiments. Moreover, we will summarize the available DeepFake dataset trends, focusing on their improvements. Additionally, the issue of how DeepFake detection aims to generate a generalized DeepFake detection model will be analyzed. Finally, the challenges related to DeepFake creation and detection will be discussed. We hope that the knowledge encompassed in this survey will accelerate the use of deep learning in face image and video DeepFake detection methods.

show abstract

“…A two-stream network is proposed for detecting image manipulations where one of the streams uses RGB image for identifying high-level tampering artifacts like contrast difference and unnatural boundaries [59]. More recently, SSTNet [52] uses a detection framework to detect tampered faces through spatial, steganalysis, and temporal features. Some methods have also used the artifacts produced in the frequency spectrum to mine better discriminative feature representations.…”

Section: A Literature Reviewmentioning

confidence: 99%

“…Following the recent works [28], [31], [35], [40], we use this metric to evaluate the cross-data performance and ablation experiments. For better comparison with recent methods [32], [35], [40], [52], we report the average metric scores for all the frames in a video. For cross-database evaluation on Celeb-DF [31], we extract 25 frames per video and report the framelevel AUC scores as suggested in the original paper.…”

Section: B Evaluation Metricsmentioning

confidence: 99%

MD-CSDNetwork: Multi-Domain Cross Stitched Network for Deepfake Detection

Agarwal¹,

Sinha²,

Vatsa³

et al. 2021

Preprint

View full text Add to dashboard Cite

The rapid progress in the ease of creating and spreading ultra-realistic media over social platforms calls for an urgent need to develop a generalizable deepfake detection technique. It has been observed that current deepfake generation methods leave discriminative artifacts in the frequency spectrum of fake images and videos. Inspired by this observation, in this paper, we present a novel approach, termed as MD-CSDNetwork, for combining the features in the spatial and frequency domains to mine a shared discriminative representation for classifying deepfakes. MD-CSDNetwork is a novel cross-stitched network with two parallel branches carrying the spatial and frequency information, respectively. We hypothesize that these multi-domain input data streams can be considered as related supervisory signals. The supervision from both branches ensures better performance and generalization. Further, the concept of cross-stitch connections is utilized where they are inserted between the two branches to learn an optimal combination of domain-specific and shared representations from other domains automatically. Extensive experiments are conducted on the popular benchmark dataset namely Face-Foreniscs++ for forgery classification. We report improvements over all the manipulation types in FaceForensics++ dataset and comparable results with state-of-the-art methods for crossdatabase evaluation on the Celeb-DF dataset and the Deepfake Detection Dataset.

show abstract

SSTNet: Detecting Manipulated Faces Through Spatial, Steganalysis and Temporal Features

Cited by 72 publications

References 16 publications

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

DeepFake Detection for Human Face Images and Videos: A Survey

MD-CSDNetwork: Multi-Domain Cross Stitched Network for Deepfake Detection

Contact Info

Product

Resources

About