Pairwise Fairness for Ranking and Regression

Abstract

We present pairwise fairness metrics for ranking models and regression models that form analogues of statistical fairness notions such as equal opportunity, equal accuracy, and statistical parity. Our pairwise formulation supports both discrete protected groups, and continuous protected attributes. We show that the resulting training problems can be efficiently and effectively solved using existing constrained optimization and robust optimization techniques developed for fair classification. Experiments illustrate the broad applicability and trade-offs of these methods.

Fair pairwise learning to rank

“…The fairness of the models is evaluated via the rND metric (Section IV-A1) and explicitly by the accuracy obtained when predicting the sensitive attribute from the representations they learn. We also report the group-dependent pairwise accuracies (Section IV-A2), which have been recently developed [31]. We explored a number of hyperparameter combinations, which we report in Table I.…”

“…Singh and Joachims [33] have instead focused on average exposure, which is defined as the average probabilities of all individuals belonging to a specific group to be ranked at the top of the list. More recently, Narasimhan et al [31] have argued for ranking fairness to be measured, in continuity with the disparate mistreatment concept in classification, as the difference in rank accuracy between different groups (Section IV-A2). Learning frameworks in fair ranking have so far been focused on post-processing methods [43], [8], [33].…”

“…The methodology can be thought of as a fair extension to a listwise ranking model. Another neural-based, fair pairwise ranking model [31] generalizes to various definitions of fairness by casting them as constrained optimization problems. In contrast, our model actively removes sensitive information from the training vectors and is able to generalize well to different fairness ranking measures.…”

“…(ii) We show how our models are able to output results which are both high in relevance and unbiased on standard datasets that are widely employed in the fairness literature. (iii) We compare our methods with a fair listwise learning to rank approach called DELTR [44], a fair classifier [9], [38], and with a constraint optimized pairwise ranker [31] and show that they are able to obtain rankings which are more fair and just as relevant or better. The paper is structured as follows.…”

A General Approach to Fairness with Optimal Transport

“…(referred to as sensitive attributes). This has motivated researchers to investigate techniques for ensuring that learned models satisfy fairness properties (Dwork et al 2012;Feldman et al 2015;Goh et al 2016;Chouldechova 2017;Corbett-Davies et al 2017;Gajane and Pechenizkiy 2017;Kusner et al 2017;Cotter et al 2018;Mitchell, Potash, and Barocas 2018;Verma and Rubin 2018;Zhang and Bareinboim 2018;Chiappa and Isaac 2019;Narasimhan et al 2020). Most often, fairness desiderata are expressed as constraints on the lower order moments or other functions of distributions corresponding to different sensitive attributes.…”