Learning Fair Representations via Rate-Distortion Maximization

Chowdhury, Somnath Basu Roy; Chaturvedi, Snigdha

doi:10.1162/tacl_a_00512

Cited by 3 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, the separability measure based on the rate-distortion function [12] is more robust to changes in feature dimensionality, as it utilizes the singular value of a feature to measure the distance between inter-class samples. Recent research has applied rate-distortion theory to explain neural network models, optimal feature learning methods, and so on [13].…”

Section: Icaita-2023mentioning

confidence: 99%

Separability Measure Supervised Network for Radar Target Recognition

Xue,

Zhang,

Qiu

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

In the radar automatic target recognition (RATR) field, radar high-resolution range profiles (HRRPs) have garnered significant attention. While traditional methods focus on extracting features having physical explanations, including power spectra, FFT magnitudes, etc, the effectiveness of these features relies heavily on personal experience and skills. In contrast, deep learning networks have shown strong competence in extracting discriminative features of HRRPs. However, the deep learning networks’ feature extraction procedure is solely based on the targets’ label information, which has almost no correlation with the feature separability. As a result, this approach can lead to poor convergence and limited recognition performance. To address this issue, we propose a Separability Measure Supervised Network (SMSN), which integrates a separability measure based on the rate-distortion function into the loss function to direct the training of the network. Comparative experiments on the airplane electromagnetic simulation HRRP dataset demonstrate that SMSN achieves higher recognition accuracy compared to the backbone networks, with significantly improved feature separability.

show abstract

Section: Icaita-2023mentioning

confidence: 99%

Separability Measure Supervised Network for Radar Target Recognition

Xue,

Zhang,

Qiu

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…Another line of work (Cheng et al 2020;Dixon et al 2018), use counterfactual data augmentation approaches to debias sentence embeddings. Recently, Chowdhury and Chaturvedi (2022) proposed a debiasing framework that makes representations from same protected attribute class uncorrelated by maximizing their ratedistortion function. Despite showcasing promise in a single domain, these frameworks fail to remain fair for out-ofdistribution data (Barrett et al 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Machine learning systems can often rely on a user's demographic information, like gender, race, and age (protected attributes), encoded in their representations (Elazar and Goldberg 2018) to make decisions, resulting in biased outcomes against certain demographic groups (Mehrabi et al 2021;Shah, Schwartz, and Hovy 2020). Numerous works try to achieve fairness through unawareness (Apfelbaum et al 2010) by debiasing model representations from protected attributes (Blodgett, Green, and O'Connor 2016;Elazar and Goldberg 2018;Elazar et al 2021;Chowdhury and Chaturvedi 2022). However, these techniques are only able to remove in-domain spurious correlations and fail to generalize to new data distributions (Barrett et al 2019).…”

Section: Introductionmentioning

confidence: 99%

Sustaining Fairness via Incremental Learning

Chowdhury

Chaturvedi

2023

AAAI

View full text Add to dashboard Cite

Machine learning systems are often deployed for making critical decisions like credit lending, hiring, etc. While making decisions, such systems often encode the user's demographic information (like gender, age) in their intermediate representations. This can lead to decisions that are biased towards specific demographics. Prior work has focused on debiasing intermediate representations to ensure fair decisions. However, these approaches fail to remain fair with changes in the task or demographic distribution. To ensure fairness in the wild, it is important for a system to adapt to such changes as it accesses new data in an incremental fashion. In this work, we propose to address this issue by introducing the problem of learning fair representations in an incremental learning setting. To this end, we present Fairness-aware Incremental Representation Learning (FaIRL), a representation learning system that can sustain fairness while incrementally learning new tasks. FaIRL is able to achieve fairness and learn new tasks by controlling the rate-distortion function of the learned representations. Our empirical evaluations show that FaIRL is able to make fair decisions while achieving high performance on the target task, outperforming several baselines.

show abstract

“…We subsequently perform backdoor adjustment based on the average treatment effect, utilizing feature reweighting . In TE-D, we leverage the rate-distortion function which controls the number of bits required to encode a set of vector representations (Chowdhury and Chaturvedi, 2022). We minimize the ratedistortion function for a non-linear projection of the features extracted from a biased pretrained model, while simultaneously minimizing the cross-entropy loss of predicting from these projected features.…”

Section: Introductionmentioning

confidence: 99%

Debiasing Multimodal Models via Causal Information Minimization

Patil,

Maharana,

Bansal

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Most existing debiasing methods for multimodal models, including causal intervention and inference methods, utilize approximate heuristics to represent the biases, such as shallow features from early stages of training or unimodal features for multimodal tasks like VQA, etc., which may not be accurate. In this paper, we study bias arising from confounders in a causal graph for multimodal data, and examine a novel approach that leverages causallymotivated information minimization to learn the confounder representations. Robust predictive features contain diverse information that helps a model generalize to out-of-distribution data. Hence, minimizing the information content of features obtained from a pretrained biased model helps learn the simplest predictive features that capture the underlying data distribution. We treat these features as confounder representations and use them via methods motivated by causal theory to remove bias from models. We find that the learned confounder representations indeed capture dataset biases and the proposed debiasing methods improve out-of-distribution (OOD) performance on multiple multimodal datasets without sacrificing in-distribution performance. Additionally, we introduce a novel metric to quantify the sufficiency of spurious features in models' predictions that further demonstrates the effectiveness of our proposed methods. 1

show abstract

Learning Fair Representations via Rate-Distortion Maximization

Cited by 3 publications

References 25 publications

Separability Measure Supervised Network for Radar Target Recognition

Separability Measure Supervised Network for Radar Target Recognition

Sustaining Fairness via Incremental Learning

Debiasing Multimodal Models via Causal Information Minimization

Contact Info

Product

Resources

About