Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos

Wöhler, Leslie; Zembaty, Martin; Castillo, Susana; Magnor, Marcus

doi:10.1145/3411764.3445627

Cited by 15 publications

(13 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These include quality differences, where the inner face appears blurred or pixelated compared to the surrounding video (Younus & Hasan, 2020), differences in skin tone or texture between the original and new face (Ajoy et al, 2021), and inconsistent blending at the contours of the face (Shao et al, 2022). Similar artifacts are reported by humans who correctly identify deepfakes (Wöhler et al, 2021). Movement discrepancies can also serve as indicators of deepfakes.…”

Section: Puppetry Deepfakes For Emotion Perception Researchmentioning

confidence: 80%

“…For example, one study showed that detection accuracy ranged from 24% for high-quality deepfakes to 71% for lower-quality deepfakes (Korshunov & Marcel, 2021). Other studies have shown similar detection ranges (Köbis et al, 2021;Tahir et al, 2021;Wöhler et al, 2021). Interestingly, correct identification of videos as true recordings ranges between 60 and 75%, which is similar to the rate at which high-quality deepfakes are incorrectly labelled as genuine (Köbis et al, 2021).…”

Section: Human Detection Of Deepfakesmentioning

confidence: 91%

“…Fortunately, factors that contribute to detection, such as blurring and asynchronous mouth movements (Thaw et al, 2021) are becoming increasingly easier to avoid as technology improves. One study showed that truncating longer deepfakes to 3 s did not impact manipulation recognition, suggesting conscious identification occurs rather early (Wöhler et al, 2021).…”

Section: Human Detection Of Deepfakesmentioning

confidence: 99%

“…Despite the high relevance of emotional expressions in human communication (Jack & Schyns, 2015), and the role of emotion in how people perceive fake news (Martel et al, 2020), few studies have examined the ability for deepfakes to accurately portray emotional expressions. Wöhler et al (2021) did so using face swaps, which are videos in which the inner face has been replaced with a deepfake of another identity. From a longer video of an interview, a 3-s clip was extracted that portrayed one of ten emotions.…”

Section: The Perception Of Deepfake Emotionsmentioning

confidence: 99%

See 3 more Smart Citations

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Becker,

Conduit,

Chouinard

et al. 2024

Behav Res

View full text Add to dashboard Cite

Video recordings accurately capture facial expression movements; however, they are difficult for face perception researchers to standardise and manipulate. For this reason, dynamic morphs of photographs are often used, despite their lack of naturalistic facial motion. This study aimed to investigate how humans perceive emotions from faces using real videos and two different approaches to artificially generating dynamic expressions – dynamic morphs, and AI-synthesised deepfakes. Our participants perceived dynamic morphed expressions as less intense when compared with videos (all emotions) and deepfakes (fearful, happy, sad). Videos and deepfakes were perceived similarly. Additionally, they perceived morphed happiness and sadness, but not morphed anger or fear, as less genuine than other formats. Our findings support previous research indicating that social responses to morphed emotions are not representative of those to video recordings. The findings also suggest that deepfakes may offer a more suitable standardized stimulus type compared to morphs. Additionally, qualitative data were collected from participants and analysed using ChatGPT, a large language model. ChatGPT successfully identified themes in the data consistent with those identified by an independent human researcher. According to this analysis, our participants perceived dynamic morphs as less natural compared with videos and deepfakes. That participants perceived deepfakes and videos similarly suggests that deepfakes effectively replicate natural facial movements, making them a promising alternative for face perception research. The study contributes to the growing body of research exploring the usefulness of generative artificial intelligence for advancing the study of human perception.

show abstract

Section: Puppetry Deepfakes For Emotion Perception Researchmentioning

confidence: 80%

Section: Human Detection Of Deepfakesmentioning

confidence: 91%

Section: Human Detection Of Deepfakesmentioning

confidence: 99%

Section: The Perception Of Deepfake Emotionsmentioning

confidence: 99%

See 2 more Smart Citations

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Becker,

Conduit,

Chouinard

et al. 2024

Behav Res

View full text Add to dashboard Cite

show abstract

“…Additionally, the generative architectures of these methods appear to only loosely preserve expression and gaze cues, which could have implications on human re-evaluation of the video for reference or training purposes. Face swapping methods, which are shown to enirely fool human viewers [26,27], seem better suited towards this use case, but analysis into their privacy/utility trade-off is limited.…”

Section: Related Workmentioning

confidence: 99%

Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy

Wilson¹,

Shic²,

Jenny³

et al. 2022

Preprint

View full text Add to dashboard Cite

With rapid advancements in image generation technology, face swapping for privacy protection has emerged as an active area of research. The ultimate benefit is improved access to video datasets, e.g. in healthcare settings. Recent literature has proposed deep network-based architectures to perform facial swaps and reported the associated reduction in facial recognition accuracy. However, there is not much reporting on how well these methods preserve the types of semantic information needed for the privatized videos to remain useful for their intended application. Our main contribution is a novel end-to-end face swapping pipeline for recorded videos of standardized assessments of autism symptoms in children. Through this design, we are the first to provide a methodology for assessing the privacy-utility trade-offs for the face swapping approach to patient privacy protection. Our methodology can show, for example, that current deep network based face swapping is bottle-necked by face detection in real world videos, and the extent to which gaze and expression information is preserved by face swaps relative to baseline privatization methods such as blurring. * This manuscript is a preprint that has not yet been peer-reviewed.

show abstract

Individual Deep Fake Recognition Skills are Affected by Viewer’s Political Orientation, Agreement with Content and Device Used

Sütterlin,

Ask,

Mägerle

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

AI-generated "deep fakes" is increasingly used by cybercriminals conducting targeted and tailored social engineering attacks, and for influencing public opinion. To raise awareness and efficiently train individuals in recognizing deep fakes, understanding individual differences in the ability to recognize them is central. Previous research suggested a close relationship between political attitudes and top-down perceptual and cognitive processing styles. In this study, we investigate the impact of political attitudes and agreement with the political message content on individual deep fake recognition skills. 163 adults (72 females = 44.2%) judged a series of video clips with politicians' statements across the political spectrum regarding their authenticity and their agreement with the message content. Half of the presented videos were fabricated via lipsync technology. In addition to agreement with each statement made, global political attitudes towards social and economic topics were assessed via the Social and Economic Conservatism Scale (SECS). There were robust negative associations between participants' general and social conservatism and their ability to recognize fabricated videos, especially when where there was agreement with the message content. Deep fakes watched on mobile phones and tablets were considerably less likely to be recognized compared to when watched on stationary computers. This is the first study to investigate and establish the association between political attitudes and interindividual differences in deep fake recognition. The study supports recently published research suggesting relationships between conservatism and perceived credibility of conspiracy theories and fake news in general. Implications for further research are discussed.

show abstract

Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos

Cited by 15 publications

References 65 publications

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli

Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy

Individual Deep Fake Recognition Skills are Affected by Viewer’s Political Orientation, Agreement with Content and Device Used

Contact Info

Product

Resources

About