The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs

Abstract

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting’s evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Persistence and fading of the cognitive and socio-emotional benefits of preschool education in a low-resource setting: Group differences and dose-dependent associations in longitudinal data from Vietnam

“…These may provide limited information about the likely benefits of a universal preschool program because many are not feasible to deliver “at scale” in low-resource settings. In addition, the efficacy of demonstration projects is substantially greater than their effectiveness under the implementation constraints that apply when a program is “scaled up” (e.g., Grantham-McGregor and Smith, 2016 ; Supplee et al, 2022 ; Sutherland et al, 2022 ). Sixth, in the majority of studies, preschool attendance is biased in ways that advantage children who attend.…”

Young mother risk-taking moderates doula home visiting impacts on parenting and toddler social-emotional development

“…Although hundreds of studies have documented positive impacts on a host of different child, parent, and family outcomes, findings are inconsistent across studies, the average size of effects across studies is modest (Supplee & Duggan, 2019), and the size of effects in a recently completed national study is small (Michalopoulos et al, 2019). Of particular concern is that positive impacts observed in studies of smaller demonstration projects with close ties to university researchers, are often not observable when models are taken to scale such as in statewide policy initiatives (Supplee et al, 2021).…”