DeepRhythm

Hua, Qi; Guo, Qing; Juefei-Xu, Felix; Xie, Xiaofei; Ма, Лей; Feng, Wei; Liu, Yang; Zhao, Jianjun

doi:10.1145/3394171.3413707

Cited by 151 publications

(16 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent researchers have put forward more innovative approaches. Heartbeat rhythms [19] detects disrupted or even broken heartbeat rhythms in fake videos, by monitoring small changes in skin color occur periodically as blood flows through the face. Face X-ray [11] innovatively added self-generated data to train the network to locate the mixed boundary, it shows mixing boundary for forged images and the absence of mixing for real images.…”

Section: Face Forgery Detectionmentioning

confidence: 99%

Frequency domain deepfake detection based on two-stream neural network

Zhang

Sun

2023

Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022)

View full text Add to dashboard Cite

Benefiting from the progress of deep learning driven generative models, face forgery technologies have rapidly become mature, thus raising public concerns about the illegal usage of these technologies. Despite the fact that fake images and videos are often unrecognizable to human eyes, recent work has found that hidden artifacts can be exposed in frequency domain. Our meth-od introduces the idea of separating human face area using landmarks before mining the forgery patterns in frequency domain. This we believe can help the network learn more discriminative features, also, a two-stream learning framework combining single-frame pathway and multi-frame pathway is developed to mine frequency clues. Compared with previous methods, our approach showed good results while using a few training data and little time. The effectiveness of our method is shown on different versions of the FF++ and Celeb-DF dataset.

show abstract

Section: Face Forgery Detectionmentioning

confidence: 99%

Frequency domain deepfake detection based on two-stream neural network

Zhang

Sun

2023

Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022)

View full text Add to dashboard Cite

show abstract

“…To enhance the robustness of detection models, it is essential to introduce more discriminative information. Some methods have attempted to use semantically biometric or inter-frame irregularity features to identify fake videos [23][24][25][26]. For instance, identifying biological anomalies (such as teeth, forehead, heart rate, etc.…”

Section: Related Workmentioning

confidence: 99%

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

The rampant use of forgery techniques poses a significant threat to the security of celebrities' identities. Although current deepfake detection methods have shown effectiveness when dealing with specific public face forgery datasets, their reliability diminishes when applied to open data. Moreover, these methods are susceptible to re-compression and mainly rely on pixel-level abnormalities in forgery faces. In this study, we present a novel approach to detecting face forgery by leveraging individual speaking patterns of facial expressions and head movements. Our method utilizes potential motion patterns and inter-frame variations to effectively differentiate between fake and real videos. We propose an end-to-end dual-branch detection network, named the spatial-temporal transformer (STT), which aims to safeguard the identity of the person-of-interest (POI) from deepfaking. The STT incorporates the spatial transformer (ST) to establish the connection between facial expressions and head movements, while the temporal transformer (TT) exploits inconsistencies in facial attribute changes. Additionally, we introduce a central compression loss to enhance the detection performance. Extensive experiments are conducted to evaluate the effectiveness of the STT, and the results demonstrate its superiority over other SOTA methods in detecting forgery videos involving POIs. Furthermore, our network exhibits resilience to pixel-level re-compression perturbations, making it a robust solution in the face of evolving forgery techniques.

show abstract

“…Qi et al [ 7 ] proposed an effective detection method utilizing remote visual photoplethysmography (PPG). Capturing and comparing the heartbeat rhythms of both the real and fake faces is the key idea of this method.…”

Section: Related Workmentioning

confidence: 99%

“…Anyone can generate these videos by combining the data available with free and open-source tools such as FaceApp [ 6 ]. Some positive applications of deepfake tools can be seen in movie productions, photography, and even video games [ 7 ]. However, deepfake technology has been infamously used for malicious purposes, such as creating fake news [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Improved Optical Flow Estimation Method for Deepfake Videos

Nassif

Nasir

Talib

et al. 2022

Sensors

View full text Add to dashboard Cite

Creating deepfake multimedia, and especially deepfake videos, has become much easier these days due to the availability of deepfake tools and the virtually unlimited numbers of face images found online. Research and industry communities have dedicated time and resources to develop detection methods to expose these fake videos. Although these detection methods have been developed over the past few years, synthesis methods have also made progress, allowing for the production of deepfake videos that are harder and harder to differentiate from real videos. This research paper proposes an improved optical flow estimation-based method to detect and expose the discrepancies between video frames. Augmentation and modification are experimented upon to try to improve the system’s overall accuracy. Furthermore, the system is trained on graphics processing units (GPUs) and tensor processing units (TPUs) to explore the effects and benefits of each type of hardware in deepfake detection. TPUs were found to have shorter training times compared to GPUs. VGG-16 is the best performing model when used as a backbone for the system, as it achieved around 82.0% detection accuracy when trained on GPUs and 71.34% accuracy on TPUs.

show abstract

DeepRhythm

Cited by 151 publications

References 48 publications

Frequency domain deepfake detection based on two-stream neural network

Frequency domain deepfake detection based on two-stream neural network

Spatial-temporal transformer network for protecting person-of-interest from deepfaking

Improved Optical Flow Estimation Method for Deepfake Videos

Contact Info

Product

Resources

About