Psychophysical Evaluation of Human Performance in Detecting Digital Face Image Manipulations

Abstract

In recent years, increasing deployment of face recognition technology in security-critical settings, such as border control or law enforcement, has led to considerable interest in the vulnerability of face recognition systems to attacks utilising legitimate documents, which are issued on the basis of digitally manipulated face images. As automated manipulation and attack detection remains a challenging task, conventional processes with human inspectors performing identity verification remain indispensable. These circumstances merit a closer investigation of human capabilities in detecting manipulated face images, as previous work in this field is sparse and often concentrated only on specific scenarios and biometric characteristics. This work introduces a web-based, remote visual discrimination experiment on the basis of principles adopted from the field of psychophysics and subsequently discusses interdisciplinary opportunities with the aim of examining human proficiency in detecting different types of digitally manipulated face images, specifically face swapping, morphing, and retouching. In addition to analysing appropriate performance measures, a possible metric of detectability is explored. Experimental data of 306 probands indicate that detection performance is widely distributed across the population and detection of certain types of face image manipulations is much more challenging than others.

A bibliometric analysis of research trends for missing persons globally and in East Asia from 2000 to 2021

“…Some of the more recent studies have provided important inputs in areas such as the potential trauma of collecting the victims’ DNA for their families [ 54 ], ethical inquiries [ 5 , 55 ], and psychological assessments [ 56 , 57 ]. An increasing body of social science research is currently available and more researchers are starting to focus on this area of research.…”

Crowd-Powered Face Manipulation Detection: Fusing Human Examiner Decisions

“…This section briefly describes the experimental setup. Conducted experiments build upon the work of [18] to which the interested reader is referred to for more details. In order to fuse decisions of human examiners, they first participated in an experiment for detecting digital face image manipulations.…”

“…Similar to automated detection systems, human performance for detecting digitally altered face images has been analysed in some previous works, e.g. in [9][10][11][12][13][14][15][16][17][18]. While research in the area of human face manipulation detection is relatively sparse, initial studies have indicated that humans display inadequate proficiency in detecting certain types of manipulated face images.…”

Overview of Research Progress of Digital Image Processing Technology

“…Nowadays, with the rapid development of various recognition technologies, more new technologies based on feature recognition are emerging. R. et al introduced a network-based remote visual recognition experiment, which aims to test human proficiency in detecting different kinds of digital image signals based on the principles adopted in psychophysics [13]. By manipulating facial images, especially face exchange, deformation, and modification, the pattern of human recognition image information can be further analyzed, which is helpful to improve the face recognition algorithm.…”