Bias in Multimodal AI: Testbed for Fair Automatic Recruitment

Peña, Alejandro; Serna, Ignacio; Morales, Aythami; Fiérrez, Julián

doi:10.1109/cvprw50498.2020.00022

Cited by 38 publications

(36 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our experiment also reveals many unconventional results that future work have to address. Our findings strongly motivate further advances in making recognition systems more robust against covariates [44], [28], explainable [5], [50], and fair [60], [51]. We hope that these findings help to develop robust and bias-mitigating face recognition solutions and help also to move forward biasaware and bias-mitigating technology in other AI application areas.…”

Section: Introductionsupporting

confidence: 52%

A Comprehensive Study on Face Recognition Biases Beyond Demographics

Terhörst

Kolf

Huber

et al. 2022

IEEE Trans. Technol. Soc.

Self Cite

View full text Add to dashboard Cite

Face recognition (FR) systems have a growing effect on critical decision-making processes. Recent works have shown that FR solutions show strong performance differences based on the user's demographics. However, to enable a trustworthy FR technology, it is essential to know the influence of an extended range of facial attributes on FR beyond demographics. Therefore, in this work, we analyse FR bias over a wide range of attributes. We investigate the influence of 47 attributes on the verification performance of two popular FR models. The experiments were performed on the publicly available MAAD-Face attribute database with over 120M high-quality attribute annotations. To prevent misleading statements about biased performances, we introduced control group based validity values to decide if unbalanced test data causes the performance differences. The results demonstrate that also many non-demographic attributes strongly affect the recognition performance, such as accessories, hair-styles and colors, face shapes, or facial anomalies. The observations of this work show the strong need for further advances in making FR system more robust, explainable, and fair. Moreover, our findings might help to a better understanding of how FR networks work, to enhance the robustness of these networks, and to develop more generalized bias-mitigating face recognition solutions.

show abstract

Section: Introductionsupporting

confidence: 52%

A Comprehensive Study on Face Recognition Biases Beyond Demographics

Terhörst

Kolf

Huber

et al. 2022

IEEE Trans. Technol. Soc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, nowadays, the use of algorithms during the recruitment process is increasing. For example, there is a growing trend to use artificial intelligence algorithms to scan resumes [30] and video-interviews [31]. Moreover, personnel selection processes also involve the use of algorithms during the earliest stages of the selection process, that is, even before the short-listing phase.…”

Section: Gender Biases In Humans Through Job Platformsmentioning

confidence: 99%

“…In sum, most of the field experiments based on the correspondence testing procedure have focused on the influence of human gender biases during the hiring process. However, the use of algorithms and job platforms in human resources departments are increasing (e.g [30]). Typically, algorithms and job platforms are broadly used during the initial personalized automated-job alerts phase (e.g.…”

Section: Gender Biases Using Algorithms For Recruitmentmentioning

confidence: 99%

Examining potential gender bias in automated-job alerts in the Spanish market

2021

View full text Add to dashboard Cite

Numerous field experiments based on the correspondence testing procedure have documented that gender bias influences personnel selection processes. Nowadays, algorithms and job platforms are used for personnel selection processes because of their supposed neutrality, efficiency, and costs savings. However, previous research has shown that algorithms can exhibit and even amplify gender bias. The present research aimed to explore a possible gender bias in automated-job alerts generated in InfoJobs, a popular job platform in Spain. Based on the correspondence testing procedure, we designed eight matched resumes in which we manipulated the gender of the candidate for two different professional sectors (female-dominated vs. male-dominated) and two different levels of age (24 vs. 38). We examined the 3,438 offers received. No significant differences were observed in the automated-job alerts received by female and male candidates as a function of occupation category, salary, and the number of long-term contracts included in the alerts. However, we found significant differences between the female-dominated and the male-dominated sectors in all the mentioned variables. Some limitations and implications of the study are discussed. The data and materials for this research are available at the Open Science Framework, https://osf.io/kptca/.

show abstract

“…There are also many instances where blind use of algorithmic models and machine learning methods have failed us. The Google Flu Trends (Butler, 2013), IBM's Watson supercomputer cancer treatment recommendations (Ross and Swetlitz, 2018), and Apple Card Algorithm (Vigdor, 2019;Pena et al, 2020) are just some of well-publicized examples. We do not think that these problems argue against either type of models; rather their co-existence and further promotion of their better use make statistics a better playground.…”

Section: Data Models and Algorithmic Models Are No Longer Easily Distinguishablementioning

confidence: 99%

Comments on "Two Cultures": What have changed over 20 years?

He¹,

Wang²

2021

Observational Studies

View full text Add to dashboard Cite

Twenty years ago Breiman (2001) called to our attention a significant cultural division in modeling and data analysis between the stochastic data models and the algorithmic models. Out of his deep concern that the statistical community was so deeply and "almost exclusively" committed to the former, Breiman warned that we were losing our abilities to solve many real world problems. Breiman was not the first, and certainly not the only statistician, to sound the alarm; we may refer to none other than John Tukey who wrote almost 60 years ago "data analysis is intrinsically an empirical science". However, the bluntness and timeliness of Breiman's article made it uniquely influential. It prepared us for the data science era and encouraged a new generation of statisticians to embrace a more broadly defined discipline.Some might argue that "The cultural division between these two statistical learning frameworks has been growing at a steady pace in recent years", to quote Mukhopadhyay and Wang (2020). In this commentary, we focus on some of the positive changes over the past 20 years and offer an optimistic outlook for our profession.

show abstract

Bias in Multimodal AI: Testbed for Fair Automatic Recruitment

Cited by 38 publications

References 17 publications

A Comprehensive Study on Face Recognition Biases Beyond Demographics

A Comprehensive Study on Face Recognition Biases Beyond Demographics

Examining potential gender bias in automated-job alerts in the Spanish market

Comments on "Two Cultures": What have changed over 20 years?

Contact Info

Product

Resources

About