Administrative Records Mask Racially Biased Policing

Abstract

Researchers often lack the necessary data to credibly estimate racial discrimination in policing. In particular, police administrative records lack information on civilians police observe but do not investigate. In this article, we show that if police racially discriminate when choosing whom to investigate, analyses using administrative records to estimate racial discrimination in police behavior are statistically biased, and many quantities of interest are unidentified—even among investigated individuals—absent strong and untestable assumptions. Using principal stratification in a causal mediation framework, we derive the exact form of the statistical bias that results from traditional estimation. We develop a bias-correction procedure and nonparametric sharp bounds for race effects, replicate published findings, and show the traditional estimator can severely underestimate levels of racially biased policing or mask discrimination entirely. We conclude by outlining a general and feasible design for future studies that is robust to this inferential snare.

No Justice, No Peace: Political Science Perspectives on the American Carceral State

“…Without such data, we would lack knowledge about the role of descriptive representation in driving criminal legal outcomes or the ways that facially neutral institutional policies can yield unequal outcomes. At the same time, important work reminds us that administrative data by their nature, and the sort of information they do and do not capture, can obscure a wide variety of biases that inform decision making by public officials (Knox et al 2020a; 2020b). Rather than undercut the scholarly advances developed from administrative records, this work highlights that the misstep lies in casting administrative records as free from the various biases that threaten the validity of all kinds of data.…”

“…Analyses leveraging administrative data expand the scope of inquiry to develop insight from the likes of traffic stop and local government finance data. Yet, emerging research also highlights methodological challenges: administrative records are embedded with their own set of biases for which researchers have not fully accounted (Knox et al 2020a; 2020b). Other work in this issue uses a novel method to incorporate the voices of the policed in such a way that it avoids researcher-induced biases.…”

What is Your Estimand? Defining the Target Quantity Connects Statistical Evidence to Theory

“…The empirical regularity supports the misleading conclusion only under identification assumptions that the node at the bottom of each Directed Acyclic Graph (DAG, Pearl 2009) does not affect both the variable that the researchers hold constant (boxed) and the outcome (at right). We draw the Fryer (2019) example from a critique by Knox et al (2020) which highlights this and other issues with the original paper. In the first row, equal use of lethal force against black individuals stopped by police may stem from the fact that being stopped is a collider: among those stopped, the behavior of blacks is likely to be less dangerous.…”

“…In our examples about demographic disparities, the problem is especially bad in two of the three cases because no data are available for one value of the collider: we only see application decisions among those who apply, and we only see whether someone is shot among those who are stopped by police Knox et al (2020). discuss this problem in greater depth.…”

On the challenges associated with the study of police use of deadly force in the United States: A response to Schwartz & Jahn

“…Further complicating matters, being stopped by police officers may be a mediator on the causal pathway between race and police use of deadly force, and the available evidence indicates that both crime reporting [ 53 – 55 ] and proactive police stops [ 56 – 59 ] differ systematically across racial groups. As such, conditioning on stops could bias analyses by (1) blocking a mediating path and (2) inducing collider stratification bias [ 58 , 60 ]. Is the solution to ignore this mediator—which is literally a necessary precondition for being killed by a police officer—and calculate rates for the entire population (most of whom are never at risk)?…”