AI can be sexist and racist — it’s time to make it fair

Zou, James; Schiebinger, Londa

doi:10.1038/d41586-018-05707-8

Cited by 529 publications

(337 citation statements)

References 3 publications

Supporting

Mentioning

296

Contrasting

Unclassified

Order By: Relevance

“…In light of its powerful transformative force and profound impact across various societal domains, AI has sparked ample debate about the principles and values that should guide its development and use 5,6 . Fears that AI might jeopardize jobs for human workers 7 , be misused by malevolent actors 8 , elude accountability or inadvertently disseminate bias and thereby undermine fairness 9 have been at the forefront of the recent scientific literature and media coverage. Several studies have discussed the topic of ethical AI [10][11][12][13] , notably in metaassessments [14][15][16] or in relation to systemic risks 17,18 and unintended negative consequences like algorithmic bias or discrimination [19][20][21] .…”

Section: Introductionmentioning

confidence: 99%

The global landscape of AI ethics guidelines

2019

View full text Add to dashboard Cite

In the last five years, private companies, research institutions as well as public sector organisations have issued principles and guidelines for ethical AI, yet there is debate about both what constitutes "ethical AI" and which ethical requirements, technical standards and best practices are needed for its realization. To investigate whether a global agreement on these questions is emerging, we mapped and analyzed the current corpus of principles and guidelines on ethical AI. Our results reveal a global convergence emerging around five ethical principles (transparency, justice and fairness, non-maleficence, responsibility and privacy), with substantive divergence in relation to how these principles are interpreted; why they are deemed important; what issue, domain or actors they pertain to; and how they should be implemented. Our findings highlight the importance of integrating guidelinedevelopment efforts with substantive ethical analysis and adequate implementation strategies. Table 1-Ethical guidelines for AI by country of issuer

show abstract

Section: Introductionmentioning

confidence: 99%

The global landscape of AI ethics guidelines

2019

View full text Add to dashboard Cite

show abstract

“…Natural language processing (NLP) algorithms have been reported to incorporate inherent bias when trained on human language . NLP techniques such as word embedding are now used to objectively evaluate gender and ethnic stereotypes in text data .…”

Section: Introductionmentioning

confidence: 99%

Investigation of bias in an epilepsy machine learning algorithm trained on physician notes

et al. 2019

View full text Add to dashboard Cite

Racial disparities in the utilization of epilepsy surgery are well documented, but it is unknown whether a natural language processing (NLP) algorithm trained on physician notes would produce biased recommendations for epilepsy presurgical evaluations. To assess this, an NLP algorithm was trained to identify potential surgical candidates using 1097 notes from 175 epilepsy patients with a history of resective epilepsy surgery and 268 patients who achieved seizure freedom without surgery (total N = 443 patients). The model was tested on 8340 notes from 3776 patients with epilepsy whose surgical candidacy status was unknown (2029 male, 1747 female, median age = 9 years; age range = 0‐60 years). Multiple linear regression using demographic variables as covariates was used to test for correlations between patient race and surgical candidacy scores. After accounting for other demographic and socioeconomic variables, patient race, gender, and primary language did not influence surgical candidacy scores (P > .35 for all). Higher scores were given to patients >18 years old who traveled farther to receive care, and those who had a higher family income and public insurance (P < .001, .001, .001, and .01, respectively). Demographic effects on surgical candidacy scores appeared to reflect patterns in patient referrals.

show abstract

“…Finally, as MFMR seeks clusters that are unaffected by confounders like population 271 structure, age or sex, it may be useful for clustering in settings where protecting certain 272 information is important for privacy or fairness [69]. In this sense, MFMR is to GMM 273 roughly as AC-PCA [70] or contrastive PCA [71] are to ordinary PCA.…”

mentioning

confidence: 99%

Reverse GWAS: Using Genetics to Identify and Model Phenotypic Subtypes

Dahl

Cai

Ko³

et al. 2018

Preprint

View full text Add to dashboard Cite

Recent and classical work has revealed biologically and medically significant subtypes in complex diseases and traits. However, relevant subtypes are often unknown, unmeasured, or actively debated, making automatic statistical approaches to subtype definition particularly valuable. We propose reverse GWAS (RGWAS) to identify and validate subtypes using genetics and multiple traits: while GWAS seeks the genetic basis of a given trait, RGWAS seeks to define trait subtypes with distinct genetic bases. Unlike existing approaches relying on off-the-shelf clustering methods, RGWAS uses a bespoke decomposition, MFMR, to model covariates, binary traits, and population structure. We use extensive simulations to show these features can be crucial for power and calibration. We validate RGWAS in practice by recovering known stress subtypes in major depressive disorder. We then show the utility of RGWAS by identifying three novel subtypes of metabolic traits. We biologically validate these metabolic subtypes with SNP-level tests and a novel polygenic test: the former recover known metabolic GxE SNPs; the latter suggests genetic heterogeneity may explain substantial missing heritability. Crucially, statins, which are widely prescribed and theorized to increase diabetes risk, have opposing effects on blood glucose across metabolic subtypes, suggesting potential have potential translational value. Author summaryComplex diseases depend on interactions between many known and unknown genetic and environmental factors. However, most studies aggregate these strata and test for associations on average across samples, though biological factors and medical interventions can have dramatically different effects on different people. Further, more-sophisticated models are often infeasible because relevant sources of heterogeneity are not generally known a priori. We introduce Reverse GWAS to simultaneously split samples into homogeneoues subtypes and to learn differences in genetic or treatment effects between subtypes. Unlike existing approaches to computational subtype October 17, 2018 1/18 identification using high-dimensional trait data, RGWAS accounts for covariates, binary disease traits and, especially, population structure; these features are each invaluable in extensive simulations. We validate RGWAS by recovering known genetic subtypes of major depression. We demonstrate RGWAS is practically useful in a metabolic study, finding three novel subtypes with both SNP-and polygenic-level heterogeneity. Importantly, RGWAS can uncover differential treatment response: for example, we show that statin, a common drug and potential type 2 diabetes risk factor, may have opposing subtype-specific effects on blood glucose.

show abstract

AI can be sexist and racist — it’s time to make it fair

Cited by 529 publications

References 3 publications

The global landscape of AI ethics guidelines

The global landscape of AI ethics guidelines

Investigation of bias in an epilepsy machine learning algorithm trained on physician notes

Reverse GWAS: Using Genetics to Identify and Model Phenotypic Subtypes

Contact Info

Product

Resources

About