Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Abstract

The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases. Geometrically, gender bias is first shown to be captured by a direction in the word embedding. Second, gender neutral words are shown to be linearly separable from gender definition words in the word embedding. Using these properties, we provide a methodology for modifying an embedding to remove gender stereotypes, such as the association between between the words receptionist and female, while maintaining desired associations such as between the words queen and female. We define metrics to quantify both direct and indirect gender biases in embeddings, and develop algorithms to "debias" the embedding. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduce gender bias in embeddings while preserving the its useful properties such as the ability to cluster related concepts and to solve analogy tasks. The resulting embeddings can be used in applications without amplifying gender bias.

Contrastive Neighborhood Alignment

“…Nonetheless, CNA provides a new means for preserving the integrity of the model by matching the feature manifold using a contrastive term that has not been previously explored. Potential Negative Societal Impacts: CNA may accidentally transfer the bias in the source feature embedding [5] to the target model. As a result it may unintentionally amplify these biases when deploying the model.…”

“…to avoid trivial solutions where every sample is isolated. Implementations: For Isomap, LLE and Hessian LLE, the only free parameter is the number of neighbors K. We set it to 10 on synthetic data and sweep over [5,10,50,100,200] on real-world datasets. For MVU and CNA, we use a 3-layer multi-layer perceptron (MLP) with Tanh activation function as the inductive projection model.…”

Weight-Biased Language across 30 Years of Australian News Reporting on Obesity: Associations with Public Health Policy

“…To address our research questions, we adopted a novel methodology for quantifying the content of obesity-related language in news media, based on word embedding analysis (Word2Vec). Word embedding [43], is a neural network approach capable of learning distributed representations of words from a set of documents. It draws upon vector space mathematics to reveal hidden biases in language.…”

“…Use of word embedding techniques to capture social and intersectional bias is gaining attention. Bolukbasi et al [43], for example, used word embedding to examine the association between occupations and gender roles in Google News articles. More recently, Arseniev-Koehler et al [46] used word embedding to extract cultural schemata about body weight from New York Times articles.…”

“…We generated four dichotomous dimensions: gender, healthiness, socioeconomic status, and stereotypes in the manner described above. For gender, anchor words were selected from the published lexicon used by Bolukbasi et al [43]. Additional terms were added that represented gendered parenting such as (mothers, fathers), (maternity, paternity), and (motherhood, fatherhood).…”