Training prior to implementing evidence-based interventions (EBIs) is essential to reach high levels of fidelity. However, the time and cost of in-person training are often barriers to implementation. Online learning offers a potential solution, though few studies examine the relationship between online training and fidelity of implementation. This study explored whether teachers trained online have similar levels of adherence, dosage, quality of delivery, and student responsiveness compared to teachers trained in-person on the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention proven to reduce substance use and violence, as part of a national dissemination project. This study involved a sample of 989 LST teachers across 114 school districts, representing 296 schools in 14 states. All teachers were first trained in LST implementation between 2016 and 2019. Hierarchical linear models were used to assess relationships between training modality and the four fidelity outcomes. Online training was associated with lower ratings of quality of delivery compared to in-person training, but no significant associations existed between online training and adherence to the curriculum, dosage, or student responsiveness. Findings from this study generally indicate that online training builds competencies important for school-based EBI implementation, while also highlighting potential shortcomings related to quality of delivery. Ensuring the inclusion of experiential learning activities (e.g., practice delivering content, receiving feedback on delivery) may be key to quality of delivery as online trainings for facilitators of school-based EBIs evolve.

As evidence-based interventions (EBIs) become more widely disseminated, fidelity of implementation (FOI) often wanes. This study explores the association between FOI and malleable variables within classrooms that could be targeted to optimize resources without compromising FOI as school-based EBIs are disseminated across real-world settings. We utilized process evaluation data from a national dissemination project of the Botvin LifeSkills Training (LST) middle school program, a universal prevention intervention shown to reduce substance use. The sample included 1,626 teachers in 371 schools across 14 states. Hierarchical linear models examined the relationship between observational measures of implementation factors and three domains of fidelity (e.g., adherence, student responsiveness, and quality of delivery). Findings suggest that curriculum modifications, student misbehavior, and shortage of time to implement the LST middle school program were factors most associated with lower FOI. Class size, access to program materials, and whether LST was delivered in a traditional classroom setting that is well-suited for instruction (versus in a less structured environment such as the school cafeteria) are less predictive. In scale-up of classroom-based universal interventions targeting behavioral health outcomes, our findings indicate that carefully vetting modifications, supporting classroom management strategies, and ensuring sufficient class time for implementation of highly interactive EBIs such as LST are important considerations. Since changes to EBIs are inevitable, efforts are needed to guide facilitators in making adjustments that improve program fit without compromising the essential intervention activities deemed necessary to produce desired outcomes.

No abstract

No abstract

The development of evidence-based interventions (EBIs) for the prevention of behavioral health problems is well supported. However, limited data exist on the sustainability of EBIs once initial support has ceased. The current study assessed, at 2 years after initial start-up support: (1) What percent of schools sustained EBI implementation? (2) To what degree did sustaining schools implement the EBI with fidelity? (3) What were the primary reasons for not sustaining the EBI? (4) What theoretical and contextual factors during initial start-up support predicted sustainment of the EBI? The study used process evaluation data from the dissemination of the Botvin LifeSkills Training (LST) middle school program in 158 school districts (including 419 schools). Fifty-one percent of districts sustained the EBI and most of these sustaining districts reported following key fidelity guidelines. Primary reasons for discontinuing centered around low teacher or administrative support and turnover. Significant predictors of sustaining LST were higher ratings of LST’s complexity, benefit, and compatibility by teachers; more positive perceptions of organizational support from administrators; and smaller proportions of Black students. Results show that EBI sustainment and fidelity of implementation post-initial startup support are feasible, though likely not for all schools. Findings also suggest that cultivating support for the EBI among staff during start-up support may be helpful for sustainment and that social determinants of a school have a complex relationship with EBI sustainment. Future research should explore true causes of differences due to race/ethnicity as well as COVID-19 effects.