Revisited: Effective use of the large body of research on the effectiveness of programs for juvenile offenders and the failure of the model programs approach

Abstract

Elliott et al.'s essay is part of what is now a series of papers in Criminology and Public Policy on contrasting ways that the evidence base on effective interventions with juvenile offenders can be used to improve juvenile justice programming. Their paper has two themes-(a) a defense of the model programs approach as promoted by the Blueprints for Healthy Youth Development evidence-based program registry, and (b) an attack on the Standardized Program Evaluation Protocol scheme for assessing programs for juvenile offenders and the meta-analysis on which it is based. I address both those themes in this commentary in the context of the broader cross-disciplinary conversation about ways to apply research evidence to improve the effectiveness of the programs used in actual practice.

Evaluating RNR-Based Targeted Treatment and Intervention Dosage in the Context of Traumatic Exposure

“…The current study was designed to assess the efficacy of best practices in evidence-based juvenile justice systems of individualized matching of services to dynamic risk through validated assessment and providing sufficient dosage of such services as demonstrated by meta-analysis (Howell et al, 2014;Lipsey, 2018Lipsey, , 2020. The uniqueness of our approach is in the consideration of cumulative trauma in these analyses by including a measure of extensive exposure (four or more ACEs) as well as examining the interaction of extensive trauma and best practices for treatment (matching and appropriate dosage).…”

“…As, or perhaps more, importantly, the current study speaks to the ability to "scale up" best practices to have state-level impact. Lipsey highlights the importance of research examining "the system level effects on outcomes for the population of juveniles served as reforms are implemented, initiatives taken, model programs adopted, SPEP implemented" (Lipsey, 2020(Lipsey, , p. 1341. The current study assessed the relevance of FDJJ policy for all youth in residential programs to match services to youth's top assessed dynamic risks and provide such services according to SPEP-defined dosages (weeks and contact hours).…”

“…The current study sample was from among the earliest years of SPEP adoption by FDJJ across residential programs, and arguably such SPEP dosage adherence is improving based on the more recent SPEP reports from that system. 6 This demonstrates the ability to get to what Lipsey deems the "ultimate EBP system is a system-level integration of multiple workable and effective practices that produce system-level positive effects on the lives of the youth served and the communities in which they reside" (Lipsey, 2020(Lipsey, , pp. 1341.…”

“…Fortunately, leveraging over 500 independent samples from 361 high quality studies specifically examining interventions with juveniles and having offending/reoffending as an outcome measure, Lipsey (2009) has developed the Standardized Program Evaluation Protocol (SPEP). 1 The meta-analysis demonstrated the most important characteristics for programs to achieve significant juvenile offending/reoffending reductions are: 1) the type of service 2) the risk level (target higher risk youth), 3) the quality/fidelity of the intervention (adhere to the model intent with trained/credentialed staff), and 4) the dosage provided (Lipsey, 2009(Lipsey, , 2014(Lipsey, , 2018(Lipsey, , 2020Lipsey et al, 2010). The SPEP sets dosage targets using the pool of studies in the meta-analysis for each service type (such as cognitive behavioral therapy, group counseling, skill building, etc.)…”

“… 1. Of note, for interested readers there are several papers available which contrast the model program approach (rating brand name programs on evidence-based registries) with the SPEP approach (leveraging meta-analysis) as discussions on how to best use the growing body of research on program effectiveness for juvenile offenders (see Elliott et al, 2020; Lipsey, 2018, 2020). The current study uses the dosage targets of the SPEP, as that was the criteria residential programs in Florida have been directed to achieve since the 2013 implementation of the SPEP by the FDJJ. …”