This study estimated reading achievement gaps in different ethnic, gender, and socioeconomic groups of 1st graders in the U.S. compared with specific reference groups and identified statistically significant correlates and moderators of early reading achievement. A subset of 2,296 students nested in 184 schools from the Early Childhood Longitudinal Study (ECLS) kindergarten to 1st-grade cohort were analyzed with hierarchical linear models. With child-level background differences controlled, significant 1st-grade reading differentials were found in African American children (Ϫ0.51 SD units below Whites), boys (Ϫ0.31 SD units below girls), and children from high-poverty households (Ϫ0.61 to Ϫ1.0 SD units below well-to-do children). In all 3 comparisons, the size of the reading gaps increased from kindergarten entry to 1st grade. Reading level at kindergarten entry was a significant child-level correlate, related to poverty status. At the school level, class size and elementary teacher certification rate were significant reading correlates in 1st grade. Cross-level interactions indicated reading achievement in African American children was moderated by the schools students attended, with attendance rates and reading time at home explaining the variance.

Federal policy tools for gathering evidence on “What Works” in education, such as the What Works Clearinghouse’s (WWC) standards, emphasize randomized field trials as the preferred method for generating scientific evidence on the effectiveness of educational programs. This article argues instead for extended-term mixed-method (ETMM) designs. Emphasizing the need to consider temporal factors in gaining thorough understandings of programs as they take hold in organizational or community settings, the article asserts that formal study of contextual and site-specific variables with multiple research methods is a necessary prerequisite to designing sound field experiments for making generalized causal inferences. A theoretical rationale and five guiding principles for ETMM designs are presented, with suggested revisions to the WWC’s standards.

Federal policy tools for gathering evidence on “What Works” in education, such as the What Works Clearinghouse’s (WWC) standards, emphasize randomized field trials as the preferred method for generating scientific evidence on the effectiveness of educational programs. This article argues instead for extended-term mixed-method (ETMM) designs. Emphasizing the need to consider temporal factors in gaining thorough understandings of programs as they take hold in organizational or community settings, the article asserts that formal study of contextual and site-specific variables with multiple research methods is a necessary prerequisite to designing sound field experiments for making generalized causal inferences. A theoretical rationale and five guiding principles for ETMM designs are presented, with suggested revisions to the WWC’s standards.

This article synthesizes research on standards-based reforms and accountability, with specific attention to purposes, models, and methods of inquiry. Starting with the premise that the reforms were meant to be systemic, the article examines the extent to which studies were guided by designs that explicitly or implicitly acknowledge a system, and evaluates the utility of the designs in generating information to support large-scale systemic changes in education. The article concludes that research efforts on reforms have been largely nonsystemic in design and have thereby failed to adequately help individual schools, school systems, and statewide systems to develop in directions that are consistent with the mission of the reform movement.

No abstract