Measuring and Mitigating Unintended Bias in Text Classification

Dixon, Lucas; Li, John; Sorensen, Jeffrey; Thain, Nithum; Vasserman, Lucy

doi:10.1145/3278721.3278729

Cited by 552 publications

(681 citation statements)

References 5 publications

Supporting

Mentioning

670

Contrasting

Order By: Relevance

“…Results are shown in Table 4. Unlike previous approaches (Park et al, 2018;Dixon et al, 2018;Madras et al, 2018), our method does not degrade classifier performance (it even improves) in terms of all reported metrics. We also look at samples containing identity terms.…”

Section: Evaluation On Original Datamentioning

confidence: 48%

Incorporating Priors with Feature Attribution on Text Classification

Liu¹,

Avci²

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Feature attribution methods, proposed recently, help users interpret the predictions of complex models. Our approach integrates feature attributions into the objective function to allow machine learning practitioners to incorporate priors in model building. To demonstrate the effectiveness our technique, we apply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in a scarce data setting by forcing the model to focus on toxic terms. Our approach adds an L 2 distance loss between feature attributions and task-specific prior values to the objective. Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a tradeoff on the original task; ii) incorporating priors helps model performance in scarce data settings.

show abstract

Section: Evaluation On Original Datamentioning

confidence: 48%

Incorporating Priors with Feature Attribution on Text Classification

Liu¹,

Avci²

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…"Wiki Madlibs" Eval Dataset [8]: For the Dataset augmentation based bias mitigation baseline strategy, we follow the approach proposed by Dixon et al [8]. For every BSW, they manually create templates for both hateful instances and neutral instances.…”

Section: Datasetsmentioning

confidence: 99%

“…In order to empirically measure the performance of the de-biasing procedures, Dixon et al [8] proposed the Pinned AUC Equality Difference (pAUC) metric to quantify the bias in learning for the dataset augmentation strategy only.…”

Section: Pinned Auc Equality Differencementioning

confidence: 99%

“…We use 100D GloVe [19] embeddings to encode text and CNN architecture with 128 filters of size 5×5×5 for each convolution layer, dropout of 0.3, max-sequence length as 250, categorical-cross-entropy as the loss function and RMSPROP as optimizer with learning-rate of 0.00005. We use the Figure 2: Model Architecture Diagram best parameter setting as described in Dixon et al [8] and train the classifiers for our task with batch size 128 and early stopping to a maximum of 20 epochs, keeping other hyperparameters constant throughout the experiments. We use the spaCy 2 implementation of POS tagger and NER tool.…”

Section: Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations

Badjatiya

Gupta

Varma

2019

The World Wide Web Conference

View full text Add to dashboard Cite

With the ever-increasing cases of hate spread on social media platforms, it is critical to design abuse detection mechanisms to proactively avoid and control such incidents. While there exist methods for hate speech detection, they stereotype words and hence suffer from inherently biased training. Bias removal has been traditionally studied for structured datasets, but we aim at bias mitigation from unstructured text data.In this paper, we make two important contributions. First, we systematically design methods to quantify the bias for any model and propose algorithms for identifying the set of words which the model stereotypes. Second, we propose novel methods leveraging knowledge-based generalizations for bias-free learning.Knowledge-based generalization provides an effective way to encode knowledge because the abstraction they provide not only generalizes content but also facilitates retraction of information from the hate speech detection classifier, thereby reducing the imbalance. We experiment with multiple knowledge generalization policies and analyze their effect on general performance and in mitigating bias. Our experiments with two real-world datasets, a Wikipedia Talk Pages dataset (WikiDetox) of size ∼96k and a Twitter dataset of size ∼24k, show that the use of knowledge-based generalizations results in better performance by forcing the classifier to learn from generalized content. Our methods utilize existing knowledge-bases and can easily be extended to other tasks.

show abstract

“…For the overall performance of these models, we will use the standard accuracy metric of multi-class classification. There are a number of metrics and criteria that offer different interpretations of model fairness (Dixon et al, 2018;Narayanan, 2018;Friedler et al, 2019;. In this work, we use the method introduced by (Hardt et al, 2016) as Equality of Opportunity.…”

Section: Evaluation Metricsmentioning

confidence: 99%

Debiasing Embeddings for Reduced Gender Bias in Text Classification

Prost¹,

Thain²,

Bolukbasi³

2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream classification tasks, using the case study of occupation classification (De-Arteaga et al., 2019). We show that traditional techniques for debiasing embeddings can actually worsen the bias of the downstream classifier by providing a less noisy channel for communicating gender information. With a relatively minor adjustment, however, we show how these same techniques can be used to simultaneously reduce bias and maintain high classification accuracy.

show abstract

Measuring and Mitigating Unintended Bias in Text Classification

Cited by 552 publications

References 5 publications

Incorporating Priors with Feature Attribution on Text Classification

Incorporating Priors with Feature Attribution on Text Classification

Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations

Debiasing Embeddings for Reduced Gender Bias in Text Classification

Contact Info

Product

Resources

About