Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain

Liu, Honggu; Li, Xiaodan; Zhou, Wenbo; Chen, Yuefeng; He, Yuan; Xue, Hui; Zhang, Weimin; Yu, Nenghai

doi:10.1109/cvpr46437.2021.00083

Cited by 320 publications

(96 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al [51] developed an adaptive frequency feature generation module to mine frequency clues in combination with metric learning for improved separability in the embedding space. Liu et al [56] combined the spatial image and phase spectrum to capture up-sampling artifacts and thereby improve the transferability of deepfake detection.…”

Section: Conventional Deepfake Detectionmentioning

confidence: 99%

“…The main reasons are the lack of data and ambiguity between deepfake categories. Recently introduced methods, which are trained on datasets [60] with multiple deepfake types, generally have the ability to recognize deepfake generation types [51,[56][57][58]60,63,64,66]. However, these methods lack generalization due to overfitting to few generation methods in the training process and thus cannot be used in practical contexts.…”

Section: Conventional Deepfake Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Robust Deepfake On Unrestricted Media: Generation And Detection

Le¹,

Nguyen²,

Yamagishi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advances in deep learning have led to substantial improvements in deepfake generation, resulting in fake media with a more realistic appearance. Although deepfake media have potential application in a wide range of areas and are drawing much attention from both the academic and industrial communities, it also leads to serious social and criminal concerns. This chapter explores the evolution of and challenges in deepfake generation and detection. It also discusses possible ways to improve the robustness of deepfake detection for a wide variety of media (e.g., in-the-wild images and videos). Finally, it suggests a focus for future fake media research.

show abstract

Section: Conventional Deepfake Detectionmentioning

confidence: 99%

Section: Conventional Deepfake Detectionmentioning

confidence: 99%

Robust Deepfake On Unrestricted Media: Generation And Detection

Le¹,

Nguyen²,

Yamagishi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Other works aim at detecting inconsistent head poses [105] or irregular eye blinking [66], although more recent fakes may not exhibit such anomalies. More recently, works have focused on attention mechanisms [35,99,110], exploiting the frequency spectrum [41,43,63,70,72,76,89,109], detecting anomalies in features from a face recognition network [101], or using extra identity information [6,33,38]. [44] and [108] use self-supervision for frame-based detection, but do not study the effect of using many real samples.…”

Section: Face Forgery Detectionmentioning

confidence: 99%

Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection

Haliassos¹,

Mira²,

Petridis³

et al. 2022

Preprint

View full text Add to dashboard Cite

One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we question whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real videos to learn, in a self-supervised crossmodal manner, temporally dense video representations that capture factors such as facial movements, expression, and identity. Second, we use these learned representations as targets to be predicted by our forgery detector along with the usual binary forgery classification task; this encourages it to base its real/fake decision on said factors. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments, and examine the factors that contribute to its performance. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.

show abstract

“…The uneven overlapping multiplication of two coordinate axes results in the image block similar to chessboard [46], and resulting in a loss of facial texture details. Liu et al [47] observe that the upsampling is a necessary step of most face forgery techniques and utilize phase spectrum to capture the up-sampling defects of face forgery. Since the up-sampling occurs between adjacent pixels, it is advantageous to capture the local information and collect statistics by using small blocks of appropriate size [48].…”

Section: Analysis Of Deepfakementioning

confidence: 99%

Detection of Deepfake Videos Using Long Distance Attention

Lu¹,

Liu²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain

Cited by 320 publications

References 32 publications

Robust Deepfake On Unrestricted Media: Generation And Detection

Robust Deepfake On Unrestricted Media: Generation And Detection

Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection

Detection of Deepfake Videos Using Long Distance Attention

Contact Info

Product

Resources

About