Evaluating the Implementation of Comprehensive School Reform and Its Impact on Growth in Student Achievement

Abstract

This study examines the relationship between the implementation of comprehensive school reform (CSR) and growth in student achievement. Survey data about CSR implementation and school-level achievement were collected in multiple years from a sample of CSR schools and compared with a sample of matchedpair schools. The sampled CSR schools adopted several promising CSR models. Findings indicate that implementation level of some components is a growth function of implementation length, with a large variation. On average, growth in mathematics and reading achievement among students at CSR schools is not consistently greater than the growth among similar students at comparison schools. However, after controlling for implementation level and limiting the implementation length to 3-5 years, overall CSR schools made larger growth in mathematics and reading achievement relative to the matched comparison schools when CSR schools had a high level of implementation or made progress in implementation level in some components over time. The relationship between CSR implementation level and growth in reading varies by CSR model.

Inspiration, Perspiration, and Time: Operations and Achievement in Edison Schools

“…Effect sizes for nontested curriculum on math and reading scores are on the order of half a standard deviation, which is at least moderate in size, and pretty substantial by the standards of education research. 5 To be sure, with simple cross-sectional correlations such as these, we cannot conclude that the relationship is Zhang, Shkolnik, and Fashola (2005) found that schools that had been implementing a comprehensive reform model for three to five years and that were rated as strong implementers achieved larger test-score gains than schools of similar vintage that were judged to be low-implementing. 5 Note that this effect size cannot be directly compared to the achievement z-score scale, which is standardized relative to a different distribution.…”

“…Research on comprehensive school reform has also demonstrated a positive relationship between length of implementation and student achievement (Slavin et al, 1994;Ross, Nunnery, and Smith, 1996;Catterall, 1995). Zhang, Shkolnik, and Fashola (2005), for example, found that schools in their third through fifth years of comprehensive school-reform-model implementation achieved higher test-score gains than schools with either more or less experience implementing the models. Ross et al (2001, p. 327) note that "leading scholars of educational change have hypothesized that finding measurable results will, at best, take between 3 and 10 years" (Fullan and Stiegelbauer, 1991;Fullan, 1999;Fullan and Miles, 1992).…”

Assistance and Accountability in Externally Managed Schools: The Case of Edison Schools, Inc.

“…The sample size for these analyses is less than the total number of case study schools because complete achievement data were not available for all case study schools.12 Similarly,Zhang, Shkolnik, and Fashola (2005) found that schools that had been implementing a comprehensive reform model for three to five years and that were rated as strong implementers achieved larger test-score gains than schools of similar vintage that were judged to be low implementing.Downloaded by [University of Kent] at 04:55 14 October 2014…”

State Takeover, School Restructuring, Private Management, and Student Achievement in Philadelphia

“…Existing research on educational management organizations (including RAND's analysis of Edison schools in Gill et al, 2005), charter schools, and comprehensive reforms has suggested that student achievement effects may become evident only after several years following implementation (Slavin et al, 1994;Ross, Nunnery, and Smith, 1996;Catterall, 1995;Sass, 2006;Booker et al, forthcoming;Bifulco and Ladd, 2006;Hanushek et al, 2005;Zhang, Shkolnik, and Fashola, 2005). Consequently, it is important to examine whether trends in achievement effects are evident over time and to distinguish one-year effects from four-year effects rather than combining them in a crude average.…”