Five sources of bias in natural language processing

Hovy, Dirk; Prabhumoye, Shrimai

doi:10.1111/lnc3.12432

Cited by 143 publications

(111 citation statements)

References 87 publications

Supporting

Mentioning

110

Contrasting

Unclassified

Order By: Relevance

“…The set of harms that can arise from the use of NLP however has become a recent concern in the area of trustworthy AI [81,91,158,168,169]. Hovy and Prabhumoye describe five sources of bias in NLP and potential ways to counteract it [170].…”

Section: Processesmentioning

confidence: 99%

Towards a standard for identifying and managing bias in artificial intelligence

Schwartz¹,

Vassilev²,

Greene³

et al. 2022

214

View full text Add to dashboard Cite

Bias is neither new nor unique to AI and it is not possible to achieve zero risk of bias in an AI system. NIST intends to develop methods for increasing assurance, GOVERNANCE and practice improvements for identifying, understanding, measuring, managing, and reducing bias. To reach this goal, techniques are needed that are flexible, can be applied across contexts regardless of industry, and are easily communicated to different stakeholder groups. To contribute to the growth of this burgeoning topic area, NIST will continue its work in measuring and evaluating computational biases, and seeks to create a hub for evaluating socio-technical factors. This will include development of formal guidance and standards, supporting standards development activities such as workshops and public comment periods for draft documents, and ongoing discussion of these topics with the stakeholder community.

show abstract

Section: Processesmentioning

confidence: 99%

Towards a standard for identifying and managing bias in artificial intelligence

Schwartz¹,

Vassilev²,

Greene³

et al. 2022

214

View full text Add to dashboard Cite

show abstract

“…We reviewed the techniques on identifying and resolving representation bias mostly in tabular data data sets. The existing research has briefly investigated these issues in other data types such as multimedia [13,26,71], text [33,49], graphs, streams [25], spatio-temporal [28], etc. Still, identification and resolving biases in visual data sets has drawn more attention from different research communities and in this section we present a review of the existing works.…”

Section: Expanding the Scope To Other Data Typesmentioning

confidence: 99%

Representation Bias in Data: A Survey on Identification and Resolution Techniques

Shahbazi¹,

Li²,

Asudeh³

et al. 2022

Preprint

View full text Add to dashboard Cite

The grand goal of data-driven decision-making is to help humans make decisions, not only easily and at scale but also wisely, accurately, and just. However, data-driven algorithms are only as good as the data they work with, while data sets, especially social data, often miss representing minorities. Representation Bias in data can happen due to various reasons ranging from historical discrimination to selection and sampling biases in the data acquisition and preparation methods. One cannot expect AI-based societal solutions to have equitable outcomes without addressing the representation bias. This paper surveys the existing literature on representation bias in the data. It presents a taxonomy to categorize the studied techniques based on multiple design dimensions and provide a side-by-side comparison of their properties. There is still a long way to fully address representation bias issues in data. The authors hope that this survey motivates researchers to approach these challenges in the future by observing existing work within their respective domains.

show abstract

“…8.1.1 Data Augmentation. Many concerns have been posed regarding modern NLP systems having been trained on potentially biased datasets, as as these biases can be perpetuated to downstream tasks and eventually society in the form of allocational harms [Hovy and Prabhumoye 2021]. Therefore, Costa-jussà and de Jorge [2020] claim that developing methods trained on balanced data is a first step to eliminating representational harms.…”

Section: Debiasing Using Data Manipulationmentioning

confidence: 99%

A Survey on Gender Bias in Natural Language Processing

Stanczak¹,

Augenstein²

2021

Preprint

View full text Add to dashboard Cite

Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural language processing. We analyse definitions of gender and its categories within social sciences and connect them to formal definitions of gender bias in NLP research. We survey lexica and datasets applied in research on gender bias and then compare and contrast approaches to detecting and mitigating gender bias. We find that research on gender bias suffers from four core limitations. 1) Most research treats gender as a binary variable neglecting its fluidity and continuity. 2) Most of the work has been conducted in monolingual setups for English or other high-resource languages. 3) Despite a myriad of papers on gender bias in NLP methods, we find that most of the newly developed algorithms do not test their models for bias and disregard possible ethical considerations of their work. 4) Finally, methodologies developed in this line of research are fundamentally flawed covering very limited definitions of gender bias and lacking evaluation baselines and pipelines. We see overcoming these limitations as a necessary development in future research.

show abstract

Five sources of bias in natural language processing

Cited by 143 publications

References 87 publications

Towards a standard for identifying and managing bias in artificial intelligence

Towards a standard for identifying and managing bias in artificial intelligence

Representation Bias in Data: A Survey on Identification and Resolution Techniques

A Survey on Gender Bias in Natural Language Processing

Contact Info

Product

Resources

About