Standardizing Biases: Selection Devices and the Quantification of Race

Abstract

Racial inequality persists despite major advances in formal, legal equality. Scholars and policymakers argue that individual biases (both explicit and implicit) combine with subjective organizational decision-making practices to perpetuate racial inequality. The standardization of decision making offers a potential solution, promising to eliminate the subjectivity that biases consequential decisions. We ask, under what conditions may standardization reduce racial inequality? Drawing on research in science studies and law and society, we argue that standardization must be understood as a heterogeneous practice capable of producing very different outcomes depending on the details of the standard and the organizational infrastructure surrounding its use. We compare selection devices—simple quantified tools for making allocation decisions—in undergraduate admissions and child welfare to highlight the complex relationships between race and standardization. Child welfare agencies adopted a colorblind actuarial device that attempted to predict which children were most at risk and then make decisions based on those predictions. In contrast, the University of Michigan’s points system explicitly considered and valued race, with the goal of increasing minority student enrollments in the context of promoting student body diversity. Comparing these cases demonstrates how actuarial standardization practices, including those adopted with the intention of reducing racial inequality, tend to reinforce an unequal status quo by ideologically reconfiguring mutable social structures into immutable individual risk factors. In contrast, nonactuarial practices that explicitly promote racial equality are vulnerable to political challenges as they violate norms of colorblindness and cannot be defended in terms of their predictive validity.

Evictions and Neighborhood Child Maltreatment Reports

“…U.S. CPS agencies, tasked with responding to child maltreatment, receive and respond to reports pertaining to more than 3 million children each year (U.S. Department of Health & Human Services [HHS], 2020). Upon receiving maltreatment reports, social workers visit reported families to assess children's safety, often with the assistance of structured decision-making tools (Hirschman & Bosk, 2020). Based on cumulative prevalence estimates from national administrative data, 37% of U.S. children experience an investigation by CPS at some point during their childhoods, following reports of child abuse and/or neglect (Kim, Wildeman, Jonson-Reid, & Drake, 2017).…”

Digital prediction technologies in the justice system: The implications of a ‘race-neutral’ agenda

“…So far, we have seen that looking beneath the mask of the liberal race neutral frame reveals the structural dynamics granting the creators of standardized prediction technologies discretionary powers to preside over key creational dynamics. To illuminate the racializing effects of these dynamics, this section draws on themes distilled from the sociology of standardization (Hirschman and Bosk, 2019). The themes reject the logics of standardization and, as noted earlier, they are mythical and reflective of what I conceptualize as the bias elimination fallacy and the scientific neutrality fallacy.…”

“…A relevant theme from the sociological scholarship on standardization is the refutation of the standardization logic or claim that removing race from the language of standardized classification systems automatically renders the systems neutral and free of bias even if structural sources of discrimination are ignored (Hirschman and Bosk, 2019;Reskin, 2012;Timmermans and Epstein, 2010). This standardization logic is flawed, and it can be conceptualized as the 'bias elimination fallacy' since, as the foregoing critique of superficial liberal frames indicates, values such as race neutrality do not automatically eliminate racially unequal outcomes.…”

“…The focus on the predictive accuracy of prediction technologies without proper structural analysis exemplifies this. It ignores the moral imperative of analysing and challenging structural inequities (Hannah-Moffat, 2016;Hirschman and Bosk, 2019).…”

“…There is therefore a presumption that the technologies reflect the logics of standardization. Central to this is the belief that excluding social categories such as race and using complex numerical analysis: eliminates bias; ensures scientific objectivity; and improves efficiency (Hirschman and Bosk, 2019;Reskin, 2012;Timmermans and Epstein, 2010). But this article uses themes from the sociology of standardization to critique these logics.…”