Evaluating the Underlying Gender Bias in Contextualized Word Embeddings

Basta, Christine; Costa-jussà, Marta R.; Casas, Noe

doi:10.48550/arxiv.1904.08783

Cited by 23 publications

(30 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Along similar lines, Basta et al (2019) noted that contextual word-embeddings are less biased than traditional word-embeddings. Yet, biases like gender are propagated heavily in downstream tasks.…”

Section: Real World Implicationsmentioning

confidence: 84%

Measuring Bias in Contextualized Word Representations

Kurita¹,

Vyas²,

Pareek³

et al. 2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

294

275

View full text Add to dashboard Cite

Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1) propose a template-based method to quantify bias in BERT;(2) show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3) conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases.

show abstract

Section: Real World Implicationsmentioning

confidence: 84%

Measuring Bias in Contextualized Word Representations

Kurita¹,

Vyas²,

Pareek³

et al. 2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

294

275

View full text Add to dashboard Cite

show abstract

“…While research on the topic of machine unlearning [Bourtoule et al 2019;Cao and Yang 2015] has started to gain traction, the problem has not yet been studied in depth for foundation models. In addition, foundation models trained on less curated internet data have been shown to exhibit harmful biases targeting specific groups (e.g., gender and racial bias) [Bender et al 2021;Basta et al 2019;Kurita et al 2019;] and can produce toxic outputs [Gehman et al 2020] ( §5.2: misuse). While strategies such as further fine-tuning the foundation model on carefully curated datasets (for potentially multiple generations) [Solaiman and Dennison 2021] or applying controllable generation techniques [Keskar et al 2019] have shown some success in mitigating harmful behavior, a framework for training equitable and safe foundation models ( §5.1: fairness) will likely require further research with a collective effort across the data collection, training, and adaptation phases as well as consultation with domain experts.…”

Section: Use Cases For Adaptationmentioning

confidence: 99%

On the Opportunities and Risks of Foundation Models

Bommasani¹,

Hudson²,

Adeli³

et al. 2021

Preprint

654

652

View full text Add to dashboard Cite

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

show abstract

“…However, their focus is different from us as our approaches aim at keeping the grammatical gender information and only removing the bias in semantic genders. A few recent studies focus on measuring and reducing gender bias in contextualized word embeddings (Zhao et al, 2019;May et al, 2019;Basta et al, 2019). However, they only focus on English embeddings in which the gender is mostly only expressed by pronouns (Stahlberg et al, 2007).…”

Section: Related Workmentioning

confidence: 99%

Examining Gender Bias in Languages with Grammatical Gender

Zhou¹,

Shi²,

Zhao³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora. However, most studies to date have focused on quantifying and mitigating such bias only in English. These analyses cannot be directly extended to languages that exhibit morphological agreement on gender, such as Spanish and French. In this paper, we propose new metrics for evaluating gender bias in word embeddings of these languages and further demonstrate evidence of gender bias in bilingual embeddings which align these languages with English. Finally, we extend an existing approach to mitigate gender bias in word embeddings under both monolingual and bilingual settings. Experiments on modified Word Embedding Association Test, word similarity, word translation, and word pair translation tasks show that the proposed approaches effectively reduce the gender bias while preserving the utility of the embeddings.

show abstract

Evaluating the Underlying Gender Bias in Contextualized Word Embeddings

Cited by 23 publications

References 13 publications

Measuring Bias in Contextualized Word Representations

Measuring Bias in Contextualized Word Representations

On the Opportunities and Risks of Foundation Models

Examining Gender Bias in Languages with Grammatical Gender

Contact Info

Product

Resources

About