Predictive Validity of Pretrial Risk Assessments: A Systematic Review of the Literature

Abstract

Bail reform is sweeping the nation and many jurisdictions are looking to pretrial risk assessment as one potential strategy to support these efforts. This article summarizes the findings of a systematic review of research examining the predictive validity of pretrial risk assessments. We reviewed 11 studies (13 publications) examining the predictive validity of six pretrial risk assessment instruments reported in the gray and peer-reviewed literature as of December, 2018. Findings typically show good to excellent predictive validity. Differences in predictive validity for men and women were mixed and small. When it could be examined, predictive validity was generally comparable across racial/ethnic subgroups; however, three comparisons revealed notably lower, albeit still fair to good, predictive validity for defendants of color than White defendants. Findings suggest that pretrial risk assessments predict pretrial outcomes with acceptable accuracy, but also emphasize the need for continued investigation of predictive validity across gender and racial/ethnic subgroups.

Use of Risk Assessments in Pretrial Supervision Decision-Making and Associated Outcomes

“…Total scores ranging from 0 to 9 classify individuals into three risk categories: Low (0-2), Moderate (3-5), and High (6+). IRAS-PAT assessments have produced predictive accuracy similar to other pretrial risk assessments (Desmarais et al, 2020;Lowder et al, 2020b). Supervision level (Low, Moderate, High) measured the judge-ordered supervision level.…”

“…To date, there are a growing number of studies focused on evaluating the accuracy and effectiveness of pretrial risk assessments as an alternative to money bail (Desmarais et al, 2020;Lowder et al, 2020a;Stevenson, 2018). However, there are fewer contemporary studies examining the use of risk assessment-informed pretrial supervision, despite nearly half of local jurisdictions reporting use of pretrial supervision and a similar proportion reporting use of pretrial risk assessments (Lattimore et al, 2020).…”

Examining the Predictive Validity of the Public Safety Assessment

“…Most reviews of pretrial risk assessment instruments are completed within local agencies; thus, there are few peer-reviewed studies examining their predictive validity (Desmarais et al, 2021). Most recently, DeMichele et al (2020) examined the PSA on a statewide data set collected in Kentucky ( N = 164,597) between July 1, 2013, and December 30, 2014.…”

“…Individuals who scored a 1 failed to appear for court or were arrested 13% of the time, followed by 29% for those who scored a 2, 42% for those who scored a 3, and 67% for those who scored a 4. Desmarais et al (2021) examined the results of 10 pretrial risk assessment instruments and found that the AUC scores typically produced fair to medium effects (Rice & Harris, 2005). Furthermore, all PSA-specific peer-reviewed literature to date has examined Kentucky data and has been funded by the creators of the PSA—Arnold Ventures (Desmarais et al, 2021); thus, the need to rigorously examine PSA data from other jurisdictions.…”

Bail Reform in the United States: The What, Why, and How of Third Wave Efforts

“…Specifically, pretrial risk assessment instruments provide structure to reduce bias and promote a transparent and evidencebased approach for estimating the likelihood someone will fail to appear in court or be rearrested during the pretrial period. Pretrial risk assessment instruments are typically quite short, completed through record review and/or in person interview, and rely on an algorithmic approach to combine item ratings into risk scores (Desmarais et al, 2020). Jurisdictions across the United States have incorporated the use of pretrial risk assessment instruments to support bail reform .…”