A Confidence-Based Approach for Balancing Fairness and Accuracy

Abstract

We study three classical machine learning algorithms in the context of algorithmic fairness: adaptive boosting, support vector machines, and logistic regression. Our goal is to maintain the high accuracy of these learning algorithms while reducing the degree to which they discriminate against individuals because of their membership in a protected group.Our first contribution is a method for achieving fairness by shifting the decision boundary for the protected group. The method is based on the theory of margins for boosting. Our method performs comparably to or outperforms previous algorithms in the fairness literature in terms of accuracy and low discrimination, while simultaneously allowing for a fast and transparent quantification of the trade-off between bias and error.Our second contribution addresses the shortcomings of the bias-error trade-off studied in most of the algorithmic fairness literature. We demonstrate that even hopelessly naive modifications of a biased algorithm, which cannot be reasonably said to be fair, can still achieve low bias and high accuracy. To help to distinguish between these naive algorithms and more sensible algorithms we propose a new measure of fairness, called resilience to random bias (RRB). We demonstrate that RRB distinguishes well between our naive and sensible fairness algorithms. RRB together with bias and accuracy provides a more complete picture of the fairness of an algorithm.

FAE: A Fairness-Aware Ensemble Framework

“…For instance, Zafar et al [5] add fairness-related constraints in the objective function of a logistic regression model to account for fairness. Post-processing methods try to modify the model's predictions or decision boundary in order to ensure fairness [10], [13], [14]. Kamiran et al [10] propose a fair decision tree learner that combines a fairness-aware splitting criterion with post-processing leaf-relabeling.…”

“…Kamiran et al [10] propose a fair decision tree learner that combines a fairness-aware splitting criterion with post-processing leaf-relabeling. Fish et al [13] adjust the decision boundary of a boosting model based on the confidence scores of the misclassified instances. Finally, class-imbalance methods aim to deal with skewed class distributions.…”

“…The user-defined threshold controls how much discrepancy between the two groups is tolerated. Predictive performance measure: The vast majority of existing works minimize the standard error rate, e.g., [5]- [7], [10], [13], which is not useful in case of class-imbalance as it mainly reflects the performance of the model in the majority class. Moreover, EQOP measure, (c.f., Equation1) which relies on the TPR difference, is oblivious to the problem of class imbalance.…”

Adaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification

“…Recent works [17,44] have occasionally proposed similar iterative methods as baselines to compare themselves to. However, our work differs in that it employs an inferred rather than a heuristic model to produce bias-related probabilities (see Section 4).…”

“…baselines employed by [17,44]) propose that weights in Eq. 7a should be proportional to classifier error.…”