Conditional Supervised Contrastive Learning for Fair Text Classification

Abstract

Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data. However, the learned representations could potentially lead to performance disparities in downstream tasks, such as increased silencing of underrepresented groups in toxicity comment classification. In light of this challenge, in this work, we study learning fair representations that satisfy a notion of fairness known as equalized odds for text classification via contrastive learning. Specifically, we first theoretically analyze the connections between learning representations with fairness constraint and conditional supervised contrastive objectives, and then propose to use conditional supervised contrastive objectives to learn fair representations for text classification. We conduct experiments on two text datasets to demonstrate the effectiveness of our approaches in balancing the trade-offs between task performance and bias mitigation among existing baselines for text classification. Furthermore, we also show that the proposed methods are stable in different hyperparameter settings.

A Novel Information Theoretic Objective to Disentangle Representations for Fair Classification

“…To compare CLINIC with previous works, we compare against adversarial training (ADV) (Elazar and Goldberg, 2018;Coavoux et al, 2018) and the recently introduced Mutual Information upper bound (Colombo et al, 2021c) (I α ) which has been shown to offer more control over the degree of disentanglement than previous estimators. We compare CLINIC with the work of (Chi et al, 2022;Shen et al, 2021;Gupta et al, 2021;Shen et al, 2022) which uses a method that estimates I(Z; S) (see Eq. 4).…”

rematchka at ArAIEval Shared Task: Prefix-Tuning & Prompt-tuning for Improved Detection of Propaganda and Disinformation in Arabic Social Media Content

“…It takes as input a pair of samples that are either similar or dissimilar, and it brings similar samples closer and dissimilar samples far apart in embedding space (Khosla et al, 2020). Such loss has shown model performance improvement compared to cross-entropy on multiple problems (Chi et al, 2022;Chen et al, 2022;Pan et al, 2022).…”

Rural Settlements Dataset in Changchun-Jilin Consolidated Metropolitan Area _2010

“…Task-specific methods adopt the strategy of debiasing in the fine-tuning stage of the downstream task, of which the downstream task is known Chi et al, 2022). One representative work is INLP (Ravfogel et al, 2020(Ravfogel et al, , 2022, which repeatedly trains a linear classifier that predicts the target concept, and then projects the representation into the null space of the classifier's weight matrix to remove the representation bias.…”