Predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) with Indigenous and Caucasian female and male adolescents on probation.

Abstract

Indigenous people and the courts have emphasized that it is important to examine whether scores from violence risk assessment tools are valid and appropriate for Indigenous youth. However, studies are scarce. Therefore, we examined the predictive validity of youth probation officers’ Structured Assessment of Violence Risk in Youth (SAVRY) ratings for 744 Canadian youth, including 299 Indigenous youth (219 male, 80 female), and 445 Caucasian youth (357 male, 88 female) in a prospective field study. The SAVRY summary risk ratings and risk total scores significantly predicted violent and any reoffending for Indigenous female and male youth with medium effect sizes. Relatively few significant differences in the predictive validity emerged for Indigenous and Caucasian youth. However, Historical, Protective, and Risk Total scores predicted any recidivism better for Caucasian males than Indigenous males. Also, Indigenous youth scored significantly higher on all risk domains than Caucasian youth. Opposite to predictions, the rates of false positives were higher for Caucasian youth than for Indigenous youth. Based on the results, the SAVRY appears to be a reasonable tool to use for assessing risk in Indigenous youth. However, assessors should take steps to ensure that they use the SAVRY in a culturally appropriate manner, such as considering cultural factors in case formulations and treatment planning as the SAVRY does not ground assessments in an understanding of factors such as colonialism. In addition, future research should examine culturally salient risk factors (e.g., discrimination) and examine potential causes of higher risk scores in Indigenous youth, particularly the role of both past and present-day colonialism.

Racist Algorithms or Systemic Problems? Risk Assessments and Racial Disparities

“…1 Eight tested actuarial instruments (C. Campbell et al, 2018; Flores et al, 2016; Lowder et al, 2019; Perrault et al, 2017; Schwalbe et al, 2004, 2007; Skeem & Lowenkamp, 2016), only one of which was a pretrial risk instrument (Cohen & Lowenkamp, 2018), and three tested SPJ instruments (Lowder et al, 2019; Muir et al, 2020; Perrault et al, 2017). Four of the 10 found significant racial differences in predictive accuracy.…”

“…Two indicated the difference favored Black youth (C. Campbell et al, 2018; Schwalbe et al, 2004), one indicated differences were present only when gender also was considered (Muir et al, 2020), one found overprediction for Hispanic adults (Cohen & Lowenkamp, 2018), and one found overprediction for Black youth, which appeared to be a result of the criminal history items (Schwalbe et al, 2007). None of the SPJ instrument studies test bias on the structured risk judgments.…”

“…Other studies have defined bias as a difference in error classification or false positive rates, meaning a risk instrument may falsely classify one group as high risk at a higher rate than another group, which could easily result in unfair, harsher treatment of the group with more misclassifications. This approach was taken by a few non-peer-reviewed studies that reported a bias against Black adults (e.g., Angwin et al, 2016; Lason et al, 2016 [both were discredited, see Flores et al, 2016]), and a few peer-reviewed studies since 2000 (e.g., Muir et al, 2020; Rembert et al, 2014; Skeem & Lowenkamp, 2020), which found differences in false positive rates were small to nonsignificant or favored the particular group of color (meaning more individuals who were not categorized as high-risk recidivated), with one exception (see Dressel & Farid, 2018).…”

Probation Officer Assessments of Risk when the Youth Look Different: Contributions of Structured Professional Judgment to Concerns About Racial Bias

“…A more controlled, random assignment study with a larger number of Latinx youth and JPOs is needed to better understand the differences identified for Latinx youth. Moreover, similar studies of youth from additional racial and ethnic groups that are representative of different juvenile justice populations are needed and are slowly increasing (e.g., Muir et al, 2020). Despite the limitations of the research design, an obvious benefit of this study that cannot be understated is its generalizability to actual probation practice.…”

“…Specifically, research has demonstrated risk ratings have equivalent predictive accuracy (as measured by comparing the area under receiver operator characteristic curves) or are equivalent in the functional form of their association with recidivism or violent behavior (as measured by an absence of a Race × Risk interaction in regression models, for example; Skeem & Lowenkamp, 2016) across racial and cultural groups. Studies of SPJ instruments have documented this equivalent prediction of outcomes between White individuals and Black youth (SAVRY; Perrault et al, 2017; Vincent et al, 2011), Hispanic youth (SAVRY, Vincent et al, 2011), Indigenous youth (Muir et al, 2020), Black adults (START, Lowder et al, 2019), and non-Caucasian adults (HCR-20V2, O’Shea et al, 2014). Moreover, using HCR-20 total risk scores created for research purposes, researchers have reported equivalent predictive accuracy for recidivism between White and Black adults (Snowden et al, 2010), and equivalent predictive accuracy for inpatient violence between White, Asian and Hawaiian adults on the HCR-20 (Fujii et al, 2005).…”