Bad apples and incredible certitude

Abstract

Research summary: Chalfin and Kaplan attend to the problem of police misconduct with a series of simulation analyses that leverage data on complaints and uses of force in the Chicago Police Department. They conclude that incapacitating officers has minimal effects on misconduct and that, given political constraints, policy makers may prefer broader reforms around accountability and management to removing "bad apples". In this comment, we argue that this conclusion and its policy implications are characterized by "incredible certitude" driven by a selective focus on a subset of their full simulation results and inadequate incorporation of network spillovers into their analysis.Policy implications: Contrary to CK's conclusions, their preferred estimates of the effect of incapacitating "bad apples" on misconduct are squarely within the range of other interventions aimed at reducing police complaints and use of force. Once network spillovers are accounted for, estimates are up to five times as large. We conclude with a discussion of how even small reductions in misconduct can have outsized benefits as measured in both dollars and human suffering, and argue that the removal of problem officers is a normative good that should be pursued on moral grounds.

Using Network Analytics to Improve Targeted Disruption of Police Misconduct

“…Identifying and distributing misconduct prevention assets to address network brokers, in conjunction with traditional methods of identification through volume of complaints, may contribute to the disruption of misconduct networks. Quispe-Torreblanca and Stewart (2019) noted misconduct prone officers as elevating misconduct among near connections by around 8% through a social contagion effect, while Sierra-Arévalo and Papachristos (2021) demonstrated that the removal of a relatively small number of officers may contribute to a substantial down-stream reduction in misconduct. It is an implication of this research, that the methodology employed here may be used to identify key problem officers in misconduct networks.…”

Police Uses of Force in the USA: a Wealth of Theories and a Lack of Evidence

“…Concluding that removing “high-complaint ‘bad apples’” from police departments would result in only small reductions in civilian complaints, Chalfin and Kaplan contend that agencies should focus their resources on broader reforms rather than committing substantial resources to EIS. Sierra-Arévalo and Papachristos ( 2021 ), in turn, challenged the analytical approach that Chalfin and Kaplan relied upon, concluding that the Chalfin and Kaplan estimates are comparable to the modest results seen in other programs. Specifically, Sierra-Arévalo and Papachristos make an argument similar to ours: the existing literature is all but devoid of empirically based recommendations that result in substantial reductions in the use of force or citizen complaints.…”

“…In the event that a policymaker agrees with Sierra-Arévalo and Papachristos’ ( 2021 ) that these “rotten apples” should be terminated, that decision may be easier to make than to implement. Investigating and terminating an officer can be difficult and time consuming, not least because of the substantive and procedural protections provided by collective bargaining agreements (Rushin, 2017 , 2019 ) or the state Law Enforcement Officers Bill of Rights.…”

Exploring the factors related to citizens' willingness to report police misconduct

“…A simulation study of use of force complaints in the Chicago Police Department (Chalfin and Kaplan, 2021) uncovered that less than one-quarter of police officers account for about two-thirds of all complaints. Chalfin and Kaplan (2021) argue that replacing the top 2% of the complaint-generating officers with less complaint-prone officers may not have a large and direct effect on the overall number of complaints, but Sierra-Arévalo and Papachristos (2021) disagree, arguing that, if these officers were not removed, the network spillover effects would be much larger.…”