Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure

Soleimany, Ava P.; Schwarting, Wilko; Bhatia, Sangeeta N.; Rus, Daniela

doi:10.1145/3306618.3314243

Cited by 201 publications

(211 citation statements)

References 19 publications

Supporting

Mentioning

211

Contrasting

Order By: Relevance

“…In addition, caution should be used particularly with the unsupervised learning components of AI given the wide availability of biased data sets and self-learning algorithms. Recent developments in bias detection and mitigation also include methods such as adopting re-sampling 118 , adversarial learning 119 , and open-source toolkits such as IBM AI Fairness 360 (AIF360) (aif360.mybluemix.net) and Aequitas (dsapp.uchicago.edu/projects/aequitas).…”

Section: Bias Detection Framework For Fairnessmentioning

confidence: 99%

Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

et al. 2020

View full text Add to dashboard Cite

Precision Medicine implies a deep understanding of inter-individual differences in health and disease that are due to genetic and environmental factors. To acquire such understanding there is a need for the implementation of different types of technologies based on artificial intelligence (AI) that enable the identification of biomedically relevant patterns, facilitating progress towards individually tailored preventative and therapeutic interventions. Despite the significant scientific advances achieved so far, most of the currently used biomedical AI technologies do not account for bias detection. Furthermore, the design of the majority of algorithms ignore the sex and gender dimension and its contribution to health and disease differences among individuals. Failure in accounting for these differences will generate sub-optimal results and produce mistakes as well as discriminatory outcomes. In this review we examine the current sex and gender gaps in a subset of biomedical technologies used in relation to Precision Medicine. In addition, we provide recommendations to optimize their utilization to improve the global health and disease landscape and decrease inequalities.

show abstract

Section: Bias Detection Framework For Fairnessmentioning

confidence: 99%

Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

et al. 2020

View full text Add to dashboard Cite

show abstract

“…AI reproduces language and bias that users and developers take with them, so if this contains racist or discriminatory slurs or concepts, the algorithms will repeat or make them more evident (Caliskan et al, 2017). Hopefully, the research on limitation of effects of biases will continue according to new interpretations (Amini et al, 2019).…”

Section: And the Previous Examples Offered By Mcdermott)mentioning

confidence: 99%

A “psychopathic” Artificial Intelligence: the possible risks of a deviating AI in Education

Zanetti

Iseppi

Cassese

2019

Research on Education and Media

View full text Add to dashboard Cite

This work analyses the use of artificial intelligence in education from an interdisciplinary point of view. New studies demonstrated that an AI can “deviate” and become potentially malicious, due to programmers’ biases, corrupted feeds or purposeful actions. Knowing the pervasive use of artificial intelligence systems, including in the educational environment, it seemed necessary to investigate when and how an AI in education could deviate. We started with an investigation of AI and the risks it poses, wondering if they could be applied also to educative AI. We then reviewed the increasing literature that deals with the use of technology in the classroom, and the criticism about it, referring to specific use cases. Finally, as a result, the authors formulate questions and suggestions for further research, to bridge conceptual gaps underlined by lack of research.

show abstract

“…Although some studies, e.g. unequal-training [9] and suppressing attributes [8,43,44,42], have made effort to mitigate racial and gender bias in several computer vision tasks, this study remains to be vacant in FR. Thus, we construct a new RFW database to facilitate the research towards this issue.…”

Section: Introductionmentioning

confidence: 99%

Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network

Wang

Deng

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

307

278

View full text Add to dashboard Cite

Racial bias is an important issue in biometric, but has not been thoroughly studied in deep face recognition. In this paper, we first contribute a dedicated dataset called Racial Faces in-the-Wild (RFW) database, on which we firmly validated the racial bias of four commercial APIs and four state-of-the-art (SOTA) algorithms. Then, we further present the solution using deep unsupervised domain adaptation and propose a deep information maximization adaptation network (IMAN) to alleviate this bias by using Caucasian as source domain and other races as target domains. This unsupervised method simultaneously aligns global distribution to decrease race gap at domain-level, and learns the discriminative target representations at cluster level. A novel mutual information loss is proposed to further enhance the discriminative ability of network output without label information. Extensive experiments on RFW, GBU, and IJB-A databases show that IMAN successfully learns features that generalize well across different races and across different databases.

show abstract

Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure

Cited by 201 publications

References 19 publications

Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

A “psychopathic” Artificial Intelligence: the possible risks of a deviating AI in Education

Racial Faces in the Wild: Reducing Racial Bias by Information Maximization Adaptation Network

Contact Info

Product

Resources

About