A Rejoinder to Dressel and Farid: New Study Finds Computer Algorithm is More Accurate than Humans at Predicting Arrest and as Good as a Group of 20 Lay Experts

Holsinger, Alexander M.; Lowenkamp, Christopher T.; Latessa, Edward J.; Serin, Ralph C.; Cohen, Thomas H.; Robinson, Charles; Flores, Anthony W.; VanBenschoten, Scott W.

doi:10.2139/ssrn.3271682

Cited by 8 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Behind these critiques, at least in part, are analyses of risk assessments completed using a single instrument in a single jurisdiction, including ProPublica’s analysis of the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) in Broward County, Florida (Angwin et al, 2016) and a subsequent study that concluded COMPAS assessments performed no better than lay persons at predicting recidivism (Dressel & Farid, 2018). Although these two particular studies have been criticized and rebutted in the academic literature (e.g., Bansak, 2019; Flores et al, 2016; Holsinger et al, 2018; Lin et al, 2020), they are among the most prominent sources cited in calls to abolish risk assessment instruments from the criminal justice system.…”

mentioning

confidence: 99%

Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature

Desmarais¹,

Zottola²,

Clarke

et al. 2020

Criminal Justice and Behavior

View full text Add to dashboard Cite

Bail reform is sweeping the nation and many jurisdictions are looking to pretrial risk assessment as one potential strategy to support these efforts. This article summarizes the findings of a systematic review of research examining the predictive validity of pretrial risk assessments. We reviewed 11 studies (13 publications) examining the predictive validity of six pretrial risk assessment instruments reported in the gray and peer-reviewed literature as of December, 2018. Findings typically show good to excellent predictive validity. Differences in predictive validity for men and women were mixed and small. When it could be examined, predictive validity was generally comparable across racial/ethnic subgroups; however, three comparisons revealed notably lower, albeit still fair to good, predictive validity for defendants of color than White defendants. Findings suggest that pretrial risk assessments predict pretrial outcomes with acceptable accuracy, but also emphasize the need for continued investigation of predictive validity across gender and racial/ethnic subgroups.

show abstract

mentioning

confidence: 99%

Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature

Desmarais¹,

Zottola²,

Clarke

et al. 2020

Criminal Justice and Behavior

View full text Add to dashboard Cite

show abstract

“…COMPAS lacks transparency, but criticisms of its performance, such as the study by Dressel and Farid (2018), are also problematic because of the way they measured the outcome (yes/no) which is not how such tools are used. Most actuarial tools provide a probability score rather than a simplistic dichotomous classification (Holsinger et al, 2018). Third, there is the wider question about the choice of what variables to include and the implications of excluding variables for the accuracy of a risk tool.…”

Section: Racial or Ethnic Profilingmentioning

confidence: 99%

Towards a More Evidence-Based Risk Assessment for People in the Criminal Justice System: the Case of OxRec in the Netherlands

Fazel

Sariaslan

Fanshawe

2022

Eur J Crim Policy Res

View full text Add to dashboard Cite

Risk assessment tools are widely used throughout the criminal justice system to assist in making decisions about sentencing, supervision, and treatment. In this article, we discuss several methodological and practical limitations associated with risk assessment tools currently in use. These include variable predictive performance due to the exclusion of important background predictors; high costs, including the need for regular staff training, in order to use many tools; development of tools using suboptimal methods and poor transparency in how they create risk scores; included risk factors being based on dated evidence; and ethical concerns highlighted by legal scholars and criminologists, such as embedding systemic biases and uncertainty about how these tools influence judicial decisions. We discuss the potential that specific predictors, such as living in a deprived neighbourhood, may indirectly select for individuals in racial or ethnic minority groups. To demonstrate how these limitations and ethical concerns can be addressed, we present the example of OxRec, a risk assessment tool used to predict recidivism for individuals in the criminal justice system. OxRec was developed in Sweden and has been externally validated in Sweden and the Netherlands. The advantages of OxRec include its predictive accuracy based on rigorous multivariable testing of predictors, transparent reporting of results and the final model (including how the probability score is derived), scoring simplicity (i.e. without the need for additional interview), and the reporting of a wide range of performance measures, including those of discrimination and calibration, the latter of which is rarely reported but a key metric. OxRec is intended to be used alongside professional judgement, as a support for decision-making, and its performance measures need to be interpreted in this light. The reported calibration of the tool in external samples clearly suggests no systematic overestimation of risk, including in large subgroups.

show abstract

“…At the same time, the Court noted that Northpointe , which had marketed COMPAS, had disputed ProPublica's analysis (Holsinger et al. 2018).…”

Section: Big Data Analytics: Lack Of Theory Explanations and Transpar...mentioning

confidence: 99%

Decoding human behavior with big data? Critical, constructive input from the decision sciences

Katsikopoulos

Canellas

2022

AI Magazine

View full text Add to dashboard Cite

Big data analytics employs algorithms to uncover people's preferences and values, and support their decision making. A central assumption of big data analytics is that it can explain and predict human behavior. We investigate this assumption, aiming to enhance the knowledge basis for developing algorithmic standards in big data analytics. First, we argue that big data analytics is by design atheoretical and does not provide process-based explanations of human behavior; thus, it is unfit to support deliberation that is transparent and explainable. Second, we review evidence from interdisciplinary decision science, showing that the accuracy of complex algorithms used in big data analytics for predicting human behavior is not consistently higher than that of simple rules of thumb. Rather, it is lower in situations such as predicting election outcomes, criminal profiling, and granting bail. Big data algorithms can be considered as candidate models for explaining, predicting, and supporting human decision making when they match, in transparency and accuracy, simple, process-based, domain-grounded theories of human behavior. Big data analytics can be inspired by behavioral and cognitive theory.

show abstract

A Rejoinder to Dressel and Farid: New Study Finds Computer Algorithm is More Accurate than Humans at Predicting Arrest and as Good as a Group of 20 Lay Experts

Cited by 8 publications

References 26 publications

Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature

Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature

Towards a More Evidence-Based Risk Assessment for People in the Criminal Justice System: the Case of OxRec in the Netherlands

Decoding human behavior with big data? Critical, constructive input from the decision sciences

Contact Info

Product

Resources

About