Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Blodgett, Su Lin; Barocas, Solon; Daumé, Hal; Wallach, Hanna

doi:10.18653/v1/2020.acl-main.485

Cited by 449 publications

(329 citation statements)

References 171 publications

Supporting

Mentioning

326

Contrasting

Unclassified

Order By: Relevance

“…We acknowledge that these are crucial considerations, and intend to incorporate them in future work. For a thorough survey and a critical discussion of best practices for researching social "biases" in NLP, including and beyond gender, see Blodgett et al (2020).…”

Section: Related Workmentioning

confidence: 99%

Multi-Dimensional Gender Bias Classification

Dinan¹,

Fan²,

Wu³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Machine learning models are trained to find patterns in data. NLP models can inadvertently learn socially undesirable patterns when training on gender biased text. In this work, we propose a novel, general framework that decomposes gender bias in text along several pragmatic and semantic dimensions: bias from the gender of the person being spoken about, bias from the gender of the person being spoken to, and bias from the gender of the speaker. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information. In addition, we collect a new, crowdsourced evaluation benchmark. Distinguishing between gender bias along multiple dimensions enables us to train better and more fine-grained gender bias classifiers. We show our classifiers are valuable for a variety of applications, like controlling for gender bias in generative models, detecting gender bias in arbitrary text, and classifying text as offensive based on its genderedness.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Dimensional Gender Bias Classification

Dinan¹,

Fan²,

Wu³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Hence, if unchecked, such representational harms in model predictions would percolate into allocational harms (cf. Crawford, 2017;Abbasi et al, 2019;Blodgett et al, 2020).…”

Section: Early Discussionmentioning

confidence: 99%

UNQOVERing Stereotyping Biases via Underspecified Questions

Тао¹,

Khashabi²,

Khot³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Warning: This paper contains examples of stereotypes that are potentially offensive.While language embeddings have been shown to have stereotyping biases, how these biases affect downstream question answering (QA) models remains unexplored. We present UN-QOVER, a general framework to probe and quantify biases through underspecified questions. We show that a naïve use of model scores can lead to incorrect bias estimates due to two forms of reasoning errors: positional dependence and question independence. We design a formalism that isolates the aforementioned errors. As case studies, we use this metric to analyze four important classes of stereotypes: gender, nationality, ethnicity, and religion. We probe five transformer-based QA models trained on two QA datasets, along with their underlying language models. Our broad study reveals that (1) all these models, with and without fine-tuning, have notable stereotyping biases in these classes; (2) larger models often have higher bias; and (3) the effect of fine-tuning on bias varies strongly with the dataset and the model size.

show abstract

“…Recently, the NLP community has focused on exploring gender bias in NLP systems (Sun et al, 2019), uncovering many gender disparities and harmful biases in algorithms and text (Cao and Chang and McKeown 2019;Costa-jussà 2019;Du et al 2019;Emami et al 2019;Garimella et al 2019;Gaut et al 2020;Habash et al 2019;Hashempour 2019;Hoyle et al 2019;Lee et al 2019a;Lepp 2019;Qian 2019;Sharifirad and Matwin 2019;Stanovsky et al 2019;O'Neil 2016;Blodgett et al 2020;Nangia et al 2020). Particular attention has been paid to uncovering, analyzing, and removing gender biases in word embeddings (Basta et al, 2019;Kaneko and Bollegala, 2019;Zhao et al, , 2018bBolukbasi et al, 2016).…”

Section: Related Workmentioning

confidence: 99%

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

Dinan

Fan

Williams

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

105

View full text Add to dashboard Cite

Social biases present in data are often directly reflected in the predictions of models trained on that data. We analyze gender bias in dialogue data, and examine how this bias is not only replicated, but is also amplified in subsequent generative chit-chat dialogue models. We measure gender bias in six existing dialogue datasets before selecting the most biased one, the multi-player textbased fantasy adventure dataset LIGHT (Urbanek et al., 2019), as a testbed for bias mitigation techniques. We consider three techniques to mitigate gender bias: counterfactual data augmentation, targeted data collection, and bias controlled training. We show that our proposed techniques mitigate gender bias by balancing the genderedness of generated dialogue utterances, and find that they are particularly effective in combination. We evaluate model performance with a variety of quantitative methods-including the quantity of gendered words, a dialogue safety classifier, and human assessments-all of which show that our models generate less gendered, but equally engaging chit-chat responses.

show abstract

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Cited by 449 publications

References 171 publications

Multi-Dimensional Gender Bias Classification

Multi-Dimensional Gender Bias Classification

UNQOVERing Stereotyping Biases via Underspecified Questions

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

Contact Info

Product

Resources

About