Out With the Old and in With the New? An Empirical Comparison of Supervised Learning Algorithms to Predict Recidivism

Abstract

Recent research has produced mixed results as to whether newer machine learning algorithms outperform older, more traditional methods such as logistic regression in predicting recidivism. In this study, we compared the performance of 12 supervised learning algorithms to predict recidivism among offenders released from Minnesota prisons. Using multiple predictive validity metrics, we assessed the performance of these algorithms across varying sample sizes, recidivism base rates, and number of predictors in the data set. The newer machine learning algorithms generally yielded better predictive validity results. LogitBoost had the best overall performance, followed by Random forests, MultiBoosting, bagged trees, and logistic model trees. Still, the gap between the best and worst algorithms was relatively modest, and none of the methods performed the best in each of the 10 scenarios we examined. The results suggest that multiple methods, including machine learning algorithms, should be considered in the development of recidivism risk assessment instruments.

The Application of Machine Learning to a General Risk–Need Assessment Instrument in the Prediction of Criminal Recidivism

“…As Duwe and Kim (2017) pointed out, and we concur, these results would seem to suggest that the type of statistical methods employed to assess risk for recidivism may depend on the purpose for which risk is being assessed and how they are being used. For example, a large correctional agency looking to automate risk classification within a geographic region or across institutions may require different technologies than an individual assessor who is looking to identify criminogenic needs and generate recommendations for case management.…”

“…Some have suggested that ML may improve the "hit rate" of extant risk-assessment tools (true positives [TPs] plus true negatives [TNs]) such as the LS scales (wormith & Bonta, 2017, p. 135), whereas others have cautioned that the incremental validity of using ML approaches may be "modest," especially when the data are less complex (e.g., a data set containing scores on a single risk-assessment instrument; Garb & wood, 2019Garb & wood, , p. 1464. However, Duwe and Kim (2017) remind us that ML includes many different statistical techniques (e.g., decision tree [DT]-based algorithms, neural networks [NN], and support vector machines [SVMs]) and applications to the criminal justice field are still in their "infancy" (p. 597). Helpful overviews of ML techniques are provided by Tollenaar and van der Heijden (2013) and Duwe and Kim (2017).…”

Optimizing predictive performance of criminal recidivism models using registration data with binary and survival outcomes

“…By applying these transformations, a nonlinear decision boundary is essentially made linear again. This absence of the linearizing transformations when logistic regression is compared to a machine learning model on data having continuous predictors is unfortunately very common in criminological comparison studies, as in [22–23, 25, 28–29, 35] or restricted to quadratic terms [36]. This absence of linearizing transformations applies to the majority of machine learning comparison studies that include linear statistical methods.…”

Effects of Automating Recidivism Risk Assessment on Reliability, Predictive Validity, and Return on Investment (ROI)

“…Berk and Bleich (2013) found that Random Forests had greater accuracy in predicting recidivism than did stochastic gradient boosting, which in turn performed slightly better than did logistic regression. Findings from several studies have concurred that machine learning algorithms offer a significant, albeit more modest, improvement in predictive performance (Breitenbach, Dieterich, Brennan, and Fan, 2009;Duwe and Kim, 2015;Hess and Turner, 2013). Little difference, however, has been found in other research between logistic regression and machine learning algorithms in predicting recidivism (Hamilton, Neuilly, Lee, and Barnoski, 2015;Liu, Yang, Ramsey, Li, and Coid, 2011;Tollenaar and Van der Heijden, 2013).…”