Professional Discretion and the Predictive Validity of a Juvenile Risk Assessment Instrument

Abstract

The ability for professionals to override the results of an actuarial risk assessment tool is an essential part of effective correctional risk classification; however, little is known about how this important function affects the predictive validity of these tools. Using data from a statewide sample of juveniles from Ohio, this study examined the impact of professional adjustments on the predictive validity of a juvenile risk assessment instrument. This study found that the original and adjusted risk levels were significant predictors of recidivism, but the original risk levels were stronger predictors of recidivism than the adjusted risk levels that accounted for overrides.

Risk Assessment Overrides: Shuffling the Risk Deck Without Any Improvements in Prediction

“…Moreover, actuarial tools are especially powerful because they provide community corrections professionals with a mechanism for making decisions about a supervisee’s risk of recidivism that is both uniform and based on standardized sets of criteria. In the absence of these standardized criteria, professionals are often left with their subjective beliefs, notions, and biases to assess a supervisee’s likelihood of committing new crimes, with the ultimate result being that professional judgment produces poorer risk predictions compared with actuarial instruments (Holsinger et al, 2001; McCafferty, 2015).…”

“…Although actuarial risk instruments outperform clinical prediction, most of these tools provide officers with an escape hatch that essentially allows them to disregard the instrument’s designated risk classification and assign their own supervision level (McCafferty, 2015). For example, a supervisee classified as low risk by an actuarial instrument could be overridden to a higher supervision level should a corrections professional decide that, in his or her judgment, the actuarial risk designation underrepresents a supervisee’s risk to reoffend (Cohen, Pendergast, & VanBenschoten, 2016).…”

“…For example, a supervisee classified as low risk by an actuarial instrument could be overridden to a higher supervision level should a corrections professional decide that, in his or her judgment, the actuarial risk designation underrepresents a supervisee’s risk to reoffend (Cohen, Pendergast, & VanBenschoten, 2016). This aspect of the risk classification process is typically referred to as professional discretion or the supervision override and is one of the key components of the risk classification process (Andrews et al, 1990; McCafferty, 2015).…”

“…Hence, supervision overrides allow officers to depart from the actuarial risk classification when the totality of a supervisee’s characteristics suggests that the person’s adjusted supervision levels should diverge from the initial risk classification. By allowing officers to take into account the totality of a supervisee’s characteristics, proponents of overrides assert that these mechanisms could potentially increase an actuarial tool’s predictive accuracy (McCafferty, 2015). A growing but still relatively limited body of research, however, suggests that overrides could have the opposite result, diminishing a risk tool’s predictive efficacy (Cohen, Pendergast, & VanBenschoten, 2016; McCafferty, 2015; Schmidt et al, 2016; Wormith et al, 2012).…”

A Rejoinder to Dressel and Farid: New Study Finds Computer Algorithm is More Accurate than Humans at Predicting Arrest and as Good as a Group of 20 Lay Experts

“…In other words, in criminal justice contexts where officers decide to ignore an actuarial risk recommendation and exercise their own discretion or judgment, the officer's decisions are usually not as predictive as that of the actuarial classification. We find it surprising that this literature was not acknowledged or discussed in this article (see Cohen, Pendergast, & VanBenschoten, 2016;McCafferty, 2017;Wormith, Hogg, & Guzzo, 2012).…”