The Success for All Model of School Reform

Making Every Study Count: Learning From Replication Failure to Improve Intervention Research

“…Yet it also sharpened the program theory of change by highlighting for whom and in what contexts SFA is most effective. For example, recent replication failures and successes of SFA (Black et al, 2009;Hanselman & Borman, 2013;Quint et al, 2015) suggest that the schoolwide program may work best in urban district and school contexts, when core literacy activities are implemented longitudinally from kindergarten to second grade for students at risk of later school failure and when school organizational supports are in place to coordinate program activities across grades and among teachers. A precise answer to the question-What works, for whom, and under what conditions?-provides useful guidance for policymakers and practitioners seeking to cost-effectively target scarce resources and maximize impacts on student outcomes.…”

“…With the rise of prominent schoolwide programs designed to improve outcomes for at-risk students (Berends et al, 2002;Cook et al, 1999;Cook et al, 2000;Quint et al, 2015), the education landscape is now replete with cluster randomized trials-experiments that assign whole schools to treatment and control conditions. In designing such trials, program developers and researchers must articulate a program theory of change that describes the causal levers that are likely to improve targeted student outcomes (C. H. Weiss, 1998).…”

“…My approach to studying the role of context in replication failure and success is to focus on examples from literacy. Experimental (and quasi-experimental) research designs have played a particularly prominent role in evaluating the impact of several large-scale literacy interventions (Borman et al, 2007;Gamse et al, 2008;May et al, 2013;Quint et al, 2015) and research-based interventions supported by reform networks such as Reading for Understanding (Douglas & Albro, 2014). As a result, the field of literacy provides several programs of intervention research that have progressed from the development of an innovation, to a tightly controlled efficacy trial, to a real-world effectiveness trial.…”