The Harms of Demographic Bias in Deep Face Recognition Research

Garcia, Raul Vicente; Wandzik, Lukasz; Grabner, Louisa; Krueger, Joerg

doi:10.1109/icb45273.2019.8987334

Cited by 37 publications

(20 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can also generate cross-ethnic faces (as participant 030 in Figure 6 ) to increase the diversity of ME dataset, which primarily consists of participants from one particular country and ethnic background [ 38 ]. It is known that demographic imbalance of the training dataset will cause external biases on the trained models [ 39 , 40 , 41 ] that results in inaccurate and erroneous predictions. With the ability to generate wide range of participants with different demographics, this can alleviate the issue.…”

Section: Resultsmentioning

confidence: 99%

Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

et al. 2021

View full text Add to dashboard Cite

Long video datasets of facial macro- and micro-expressions remains in strong demand with the current dominance of data-hungry deep learning methods. There are limited methods of generating long videos which contain micro-expressions. Moreover, there is a lack of performance metrics to quantify the generated data. To address the research gaps, we introduce a new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality. For synthetic long video generation, we use the state-of-the-art generative adversarial network style transfer method—StarGANv2. Using StarGANv2 pre-trained on the CelebA dataset, we transfer the style of a reference image from SAMM long videos (a facial micro- and macro-expression long video dataset) onto a source image of the FFHQ dataset to generate a synthetic dataset (SAMM-SYNTH). We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace. For quantitative measurement, our findings show high correlation on two Action Units (AUs), i.e., AU12 and AU6, of the original and synthetic data with a Pearson’s correlation of 0.74 and 0.72, respectively. This is further supported by evaluation method proposed by OpenFace on those AUs, which also have high scores of 0.85 and 0.59. Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements. With this article, we publish our dataset to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.

show abstract

Section: Resultsmentioning

confidence: 99%

Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Ethnicity is a salient example, with findings indicating variability in face recognition algorithms for faces of different ethnicities (O'Toole et al, 2012), with studies demonstrating poorer performance on different demographic cohorts (Klare et al, 2012). This issue has received significant attention in computer vision (Abdurrahim et al, 2018;Garcia et al, 2019), and is an active area of research (Wang & Deng, 2020). While face recognition networks are comprised of multiple steps, the algorithms that find and place landmarks may be a generator of these biases.…”

Section: Study Two -Testing For Potential Biases In Automatic Landmarmentioning

confidence: 99%

Facial metrics generated from manually and automatically placed image landmarks are highly correlated

Jones

Schild

Jones

2021

Evolution and Human Behavior

View full text Add to dashboard Cite

Research on social judgments of faces often investigates relationships between measures of face shape taken from images (facial metrics), and either perceptual ratings of the faces on various traits (e.g., attractiveness) or characteristics of the photographed individual (e.g., their health). A barrier to carrying out this research using large numbers of face images is the time it takes to manually position the landmarks from which these facial metrics are derived. Although research in face recognition has led to the development of algorithms that can automatically position landmarks on face images, the utility of such methods for deriving facial metrics commonly used in research on social judgments of faces has not yet been established. Thus, across two studies, we investigated the correlations between four facial metrics commonly used in social perception research (sexual dimorphism, distinctiveness, bilateral asymmetry, and facial width to height ratio) when measured from manually and automatically placed landmarks. In the first study, in two independent sets of open access face images, we found that facial metrics derived from manually and automatically placed landmarks were typically highly correlated, in both raw and Procrustes-fitted representations. In study two, we investigated the potential for automatic landmark placement to differ between White and East Asian faces. We found that two metrics, facial width to height ratio and sexual dimorphism, were better approximated by automatic landmarks in East Asian faces. However, this difference was small, and easily corrected with outlier detection. These data validate the use of automatically placed landmarks for calculating facial metrics to use in VALIDATING AUTOMATIC FACIAL LANDMARKS 3 research on social judgments of faces, but we urge caution in their use. We also provide a tutorial for the automatic placement of landmarks on face images.

show abstract

“…Identifying and quantifying the amount of bias in FR technology are the initial step toward less-biased FR. Garcia et al showed that face matching confidence of FR models correlates with gender and ethnicity, thus revealing demographic bias [37]. Cavazos et al demonstrated that different thresholds are needed to equalize false accept rates (FARs) and the recognition accuracy [38].…”

Section: Related Workmentioning

confidence: 99%

Bias, awareness, and ignorance in deep-learning-based face recognition

et al. 2021

View full text Add to dashboard Cite

Face Recognition (FR) is increasingly influencing our lives: we use it to unlock our phones; police uses it to identify suspects. Two main concerns are associated with this increase in facial recognition: (1) the fact that these systems are typically less accurate for marginalized groups, which can be described as “bias”, and (2) the increased surveillance through these systems. Our paper is concerned with the first issue. Specifically, we explore an intuitive technique for reducing this bias, namely “blinding” models to sensitive features, such as gender or race, and show why this cannot be equated with reducing bias. Even when not designed for this task, facial recognition models can deduce sensitive features, such as gender or race, from pictures of faces—simply because they are trained to determine the “similarity” of pictures. This means that people with similar skin tones, similar hair length, etc. will be seen as similar by facial recognition models. When confronted with biased decision-making by humans, one approach taken in job application screening is to “blind” the human decision-makers to sensitive attributes such as gender and race by not showing pictures of the applicants. Based on a similar idea, one might think that if facial recognition models were less aware of these sensitive features, the difference in accuracy between groups would decrease. We evaluate this assumption—which has already penetrated into the scientific literature as a valid de-biasing method—by measuring how “aware” models are of sensitive features and correlating this with differences in accuracy. In particular, we blind pre-trained models to make them less aware of sensitive attributes. We find that awareness and accuracy do not positively correlate, i.e., that bias$$\ne$$ ≠ awareness. In fact, blinding barely affects accuracy in our experiments. The seemingly simple solution of decreasing bias in facial recognition rates by reducing awareness of sensitive features does thus not work in practice: trying to ignore sensitive attributes is not a viable concept for less biased FR.

show abstract

The Harms of Demographic Bias in Deep Face Recognition Research

Cited by 37 publications

References 15 publications

Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

Facial metrics generated from manually and automatically placed image landmarks are highly correlated

Bias, awareness, and ignorance in deep-learning-based face recognition

Contact Info

Product

Resources

About