Reducing Disparate Exposure in Ranking: A Learning To Rank Approach

Zehlike, Meike; Castillo, Carlos

doi:10.1145/3366424.3380048

Cited by 145 publications

(109 citation statements)

References 17 publications

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…Beutel et al [2] approach fair ranking by conducting pairwise analysis of user engagement with the protected groups in a ranking. Zehlike and Castillo [48] propose a supervised learning to rank method to optimize for fair exposure but focus only on the top position in ranking. It is not obvious how their proposed approach can be extended beyond the first rank position.…”

Section: Fairnessmentioning

confidence: 99%

Evaluating Stochastic Rankings with Expected Exposure

Dı́az

Mitra

Ekstrand

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

131

140

View full text Add to dashboard Cite

We introduce the concept of expected exposure as the average attention ranked items receive from users over repeated samples of the same query. Furthermore, we advocate for the adoption of the principle of equal expected exposure: given a fixed information need, no item should receive more or less expected exposure than any other item of the same relevance grade. We argue that this principle is desirable for many retrieval objectives and scenarios, including topical diversity and fair ranking. Leveraging user models from existing retrieval metrics, we propose a general evaluation methodology based on expected exposure and draw connections to related metrics in information retrieval evaluation. Importantly, this methodology relaxes classic information retrieval assumptions, allowing a system, in response to a query, to produce a distribution over rankings instead of a single fixed ranking. We study the behavior of the expected exposure metric and stochastic rankers across a variety of information access conditions, including ad hoc retrieval and recommendation. We believe that measuring and optimizing expected exposure metrics using randomization opens a new area for retrieval algorithm development and progress. CCS CONCEPTS • Information systems → Evaluation of retrieval results; Learning to rank.

show abstract

Section: Fairnessmentioning

confidence: 99%

Evaluating Stochastic Rankings with Expected Exposure

Dı́az

Mitra

Ekstrand

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

131

140

View full text Add to dashboard Cite

show abstract

“…There has been some recent work on improving fairness of spatial crime forecasting algorithms (Wheeler 2019;Mohler et al 2018) where a fairness penalty is added to the optimization algorithm. Future research may focus on incorporating fairness into learning to rank models of crime, similar to methods that incorporate fairness into learning to rank for information retrieval (Zehlike and Castillo 2018).…”

Section: Figmentioning

confidence: 99%

Learning to rank spatio-temporal event hotspots

et al. 2020

View full text Add to dashboard Cite

Background: Crime, traffic accidents, terrorist attacks, and other space-time random events are unevenly distributed in space and time. In the case of crime, hotspot and other proactive policing programs aim to focus limited resources at the highest risk crime and social harm hotspots in a city. A crucial step in the implementation of these strategies is the construction of scoring models used to rank spatial hotspots. While these methods are evaluated by area normalized Recall@k (called the predictive accuracy index), models are typically trained via maximum likelihood or rules of thumb that may not prioritize model accuracy in the top k hotspots. Furthermore, current algorithms are defined on fixed grids that fail to capture risk patterns occurring in neighborhoods and on road networks with complex geometries. Results:We introduce CrimeRank, a learning to rank boosting algorithm for determining a crime hotspot map that directly optimizes the percentage of crime captured by the top ranked hotspots. The method employs a floating grid combined with a greedy hotspot selection algorithm for accurately capturing spatial risk in complex geometries. We illustrate the performance using crime and traffic incident data provided by the Indianapolis Metropolitan Police Department, IED attacks in Iraq, and data from the 2017 NIJ Real-time crime forecasting challenge. Conclusion: Our learning to rank strategy was the top performing solution (PAI metric) in the 2017 challenge. We show that CrimeRank achieves even greater gains when the competition rules are relaxed by removing the constraint that grid cells be a regular tessellation.

show abstract

“…These include both unsupervised criteria that require the average exposure near the top of the ranked list to be equal for different groups [e.g. Celis, Straszak, and Vishnoi;Zehlike and Castillo 2018;, and supervised criteria that require the average exposure for a group to be proportional to the average relevance of that group's results to the query (Biega, Gummadi, and Weikum 2018;Singh and Joachims 2018;. Of these, some provide post-processing algorithms for re-ranking a given ranking (Biega, Gummadi, and Weikum 2018;Celis, Straszak, and Vishnoi 2018;Singh and Joachims 2018;, while others, like us, learn a ranking model from scratch (Zehlike and Castillo 2018;Singh and Joachims 2019).…”

Section: Related Workmentioning

confidence: 99%

“…This is an application where the unconstrained algorithm does better for the minority protected group. We use the same features asZehlike and Castillo (2018) to represent how well each topic matches each candidate; this includes a set of five aggregate features derived from word counts and tf-idf scores, and the gender protected attribute.For this task, we learn a linear model and impose a cross-group equal opportunity constraint: |A F emale>M ale − A M ale>F emale | ≤ 0.01. For robust optimization, we maximize min{A F emale>M ale , A M ale>F emale , AUC}.…”

mentioning

confidence: 99%

Pairwise Fairness for Ranking and Regression

Narasimhan

Cotter

Gupta

et al. 2020

AAAI

View full text Add to dashboard Cite

We present pairwise fairness metrics for ranking models and regression models that form analogues of statistical fairness notions such as equal opportunity, equal accuracy, and statistical parity. Our pairwise formulation supports both discrete protected groups, and continuous protected attributes. We show that the resulting training problems can be efficiently and effectively solved using existing constrained optimization and robust optimization techniques developed for fair classification. Experiments illustrate the broad applicability and trade-offs of these methods.

show abstract

Reducing Disparate Exposure in Ranking: A Learning To Rank Approach

Cited by 145 publications

References 17 publications

Evaluating Stochastic Rankings with Expected Exposure

Evaluating Stochastic Rankings with Expected Exposure

Learning to rank spatio-temporal event hotspots

Pairwise Fairness for Ranking and Regression

Contact Info

Product

Resources

About