FairALM: Augmented Lagrangian Method for Training Fair Models with Little Regret

Lokhande, Vishnu Suresh; Akash, Aditya Kumar; Ravi, Sathya N.; Singh, Vikas

doi:10.48550/arxiv.2004.01355

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fairness is becoming an important issue to consider in the design of learning algorithms. A common strategy to make an algorithm fair is to remove the influence of one/more protected attributes when training the models, see [28]. Most methods assume that the labels of protected attributes are known during training but this may not always be possible.…”

Section: Background and Motivationmentioning

confidence: 99%

“…There are up to 40 labels, each of which is binary-valued. Here, we follow [28] to focus on the attactiveness attribute (which we want to train a classifier to predict) and the gender is treated as "protected" since it may lead to an unfair classifier according to [28].…”

Section: Background and Motivationmentioning

confidence: 99%

“…Specifically, applying the loss on early layers like conv0 and conv1 gets better performance than applying at a relatively late layer like conv2. Another promising aspect of our approach is that is can easily handle the case where the protected attribute is a continuous variable (as long as a well trained regression network on the protected attribute is given) while other methods like [28,47] need to first discretize the variable and then enforce constraints which can be much more involved.…”

Section: Accuracymentioning

confidence: 99%

See 2 more Smart Citations

An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

Meng

Chakraborty

Singh

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present an efficient stochastic algorithm (RSG+) for canonical correlation analysis (CCA) using a reparametrization of the projection matrices. We show how this reparametrization (into structured matrices), simple in hindsight, directly presents an opportunity to repurpose/adjust mature techniques for numerical optimization on Riemannian manifolds. Our developments nicely complement existing methods for this problem which either require O(d 3 ) time complexity per iteration with O( 1 √ t ) convergence rate (where d is the dimensionality) or only extract the top 1 component with O( 1t ) convergence rate. In contrast, our algorithm offers a strict improvement for this classical problem: it achieves O(d 2 k) runtime complexity per iteration for extracting the top k canonical components with O( 1 t ) convergence rate. While the paper primarily focuses on the formulation and technical analysis of its properties, our experiments show that the empirical behavior on common datasets is quite promising. We also explore a potential application in training fair models where the label of protected attribute is missing or otherwise unavailable.

show abstract

Section: Background and Motivationmentioning

confidence: 99%

Section: Background and Motivationmentioning

confidence: 99%

Section: Accuracymentioning

confidence: 99%

See 1 more Smart Citation

An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

Meng

Chakraborty

Singh

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These tasks involve understanding invariance properties of data representations and/or parameters of the model we wish to learn. While mechanisms to control for extraneous variables are not strictly necessary in typical supervised learning tasks, where one focuses on predictive accuracy, over the last few years, many results have indicated how it can be quite useful (Lokhande et al 2020). For instance, controlling the influence of a protected attribute such as race or gender on a response variable such as credit worthiness enables the design…”

mentioning

confidence: 99%

Learning Invariant Representations using Inverse Contrastive Loss

Akash

Lokhande

Ravi

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

Learning invariant representations is a critical first step in a number of machine learning tasks. A common approach is given by the so-called information bottleneck principle in which an application dependent function of mutual information is carefully chosen and optimized. Unfortunately, in practice, these functions are not suitable for optimization purposes since these losses are agnostic of the metric structure of the parameters of the model. In our paper, we introduce a class of losses for learning representations that are invariant to some extraneous variable of interest by inverting the class of contrastive losses, i.e., inverse contrastive loss (ICL). We show that if the extraneous variable is binary, then optimizing ICL is equivalent to optimizing a regularized MMD divergence. More generally, we also show that if we are provided a metric on the sample space, our formulation of ICL can be decomposed into a sum of convex functions of the given distance metric. Our experimental results indicate that models obtained by optimizing ICL achieve significantly better invariance to the extraneous variable for a fixed desired level of accuracy. In a variety of experimental settings, we show applicability of ICL for learning invariant representations for both continuous and discrete protected/extraneous variables. The project page with code is available at https://github.com/adityakumarakash/ICL

show abstract

FairALM: Augmented Lagrangian Method for Training Fair Models with Little Regret

Cited by 2 publications

References 0 publications

An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

Learning Invariant Representations using Inverse Contrastive Loss

Contact Info

Product

Resources

About