Selection Problems in the Presence of Implicit Bias

Abstract

Over the past two decades, the notion of implicit bias has come to serve as an important component in our understanding of discrimination in activities such as hiring, promotion, and school admissions. Research on implicit bias posits that when people evaluate others -for example, in a hiring context -their unconscious biases about membership in particular groups can have an effect on their decision-making, even when they have no deliberate intention to discriminate against members of these groups. A growing body of experimental work has pointed to the effect that implicit bias can have in producing adverse outcomes.Here we propose a theoretical model for studying the effects of implicit bias on selection decisions, and a way of analyzing possible procedural remedies for implicit bias within this model. A canonical situation represented by our model is a hiring setting: a recruiting committee is trying to choose a set of finalists to interview among the applicants for a job, evaluating these applicants based on their future potential, but their estimates of potential are skewed by implicit bias against members of one group. In this model, we show that measures such as the Rooney Rule, a requirement that at least one of the finalists be chosen from the affected group, can not only improve the representation of this affected group, but also lead to higher payoffs in absolute terms for the organization performing the recruiting. However, identifying the conditions under which such measures can lead to improved payoffs involves subtle trade-offs between the extent of the bias and the underlying distribution of applicant characteristics, leading to novel theoretical questions about order statistics in the presence of probabilistic side information.

Fairness in Recommendation Ranking through Pairwise Comparisons

“…More distant is research on statistical parity in recommenders, which argues that in some applications items should be shown at the same rate across groups [55]. Diversity [52,31,39,46], filter bubbles [4], and feedback loops [27], while related to machine learning fairness, are not the focus of this paper.…”

Learning Fair Policies for Multi-Stage Selection Problems from Observational Data

“…Selection Problems. Kleinberg and Raghavan (2018) study selection problem with implicit bias and analyze the Rooney Rule in the selection process. They show that this rule can not only improve the representation of the disadvantaged group but also lead to higher payoffs for the decisionmaker.…”

The Frontiers of Fairness in Machine Learning

“…There is also some work on fairness in machine learning in other settings -for example, ranking [YS17,CSV17], selection [KRW17,KR18], personalization [CV17], bandit learning [JKM + 18, LRD + 17], human-classifier hybrid decision systems [MPZ17], and reinforcement learning [JJK + 17, DTB17]. But outside of classification, the literature is relatively sparse.…”