Maximizing Marginal Fairness for Dynamic Learning to Rank

Abstract

Rankings, especially those in search and recommendation systems, often determine how people access information and how information is exposed to people. Therefore, how to balance the relevance and fairness of information exposure is considered as one of the key problems for modern IR systems. As conventional ranking frameworks that myopically sorts documents with their relevance will inevitably introduce unfair result exposure, recent studies on ranking fairness mostly focus on dynamic ranking paradigms where result rankings can be adapted in real-time to support fairness in groups (i.e., races, genders, etc.). Existing studies on fairness in dynamic learning to rank, however, often achieve the overall fairness of document exposure in ranked lists by significantly sacrificing the performance of result relevance and fairness on the top results.To address this problem, we propose a fair and unbiased ranking method named Maximal Marginal Fairness (MMF). The algorithm integrates unbiased estimators for both relevance and merit-based fairness while providing an explicit controller that balances the selection of documents to maximize the marginal relevance and fairness in top-k results. Theoretical and empirical analysis shows that, with small compromises on long list fairness, our method achieves superior efficiency and effectiveness comparing to the state-of-the-art algorithms in both relevance and fairness for top-k rankings. CCS CONCEPTS• Information systems → Learning to rank.

Toward Pareto Efficient Fairness-Utility Trade-off in Recommendation through Reinforcement Learning

“…Secondly, in-processing methods aim at encoding fairness as part of the objective function, typically as a regularizer [1,4]. Finally, post-processing methods tend to modify the presentations of the results, e.g., re-ranking through linear programming [25,43,53] or multi-armed bandit [5]. However, there is no free lunch, imposing fairness constraints to the main learning task introduces a trade-off between these objectives, which have been asserted in several studies [22,23,32,55], e.g., Dutta et al [12] showed that because of noise on the underrepresented groups the trade-off between accuracy and equality of opportunity exists.…”

Fair ranking: a critical review, challenges, and future directions

“…However, such scores are rarely available in real-world settings. For example, machine-learned models or other statistical techniques used to estimate relevance scores are often uncertain about the relevance of items due to various reasons, for example, biased or noisy feedback, the initial unavailability of data [119,164], and platform updates in dynamic settings [126]. While such estimation noise (or bias) is important for all algorithmic ranking or recommendations challenges, it is especially important to consider for fair ranking algorithms, as we illustrate below.…”

Explainable Fairness in Recommendation

“…Secondly, in-processing methods aim at encoding fairness as part of the objective function, typically as a regularizer [1,6,24,41]. Finally, post-processing methods modify the presentation of the results, e.g., by re-ranking through linear programming [40,55,68] or multi-armed bandit [8]. Based on the characteristics of the recommender system itself, there also have been a few works related to multi-sided fairness in multi-stakeholder systems [7,22].…”