No abstract

Measurement scholars have recently constructed validity arguments in support of a variety of educational assessments, including classroom observation instruments. In this article, we note that users must examine the robustness of validity arguments to variation in the implementation of these instruments. We illustrate how such an analysis might be used to assess a validity argument constructed for the Mathematical Quality of Instruction instrument, focusing in particular on the effects of varying the rater pool, subject matter content, observation procedure, and district context. Variation in the subject matter content of lessons did not affect rater agreement with master scores, but the evaluation of other portions of the validity argument varied according to the composition of the rater pool, observation procedure, and district context. These results demonstrate the need for conducting such analyses, especially for classroom observation instruments that are subject to multiple sources of variation.

Taking algebra by eighth grade is considered an important milestone on the pathway to college readiness. We highlight a collaboration to investigate one district's effort to increase middle school algebra course-taking. In 2010, the Wake County Public Schools began assigning middle school students to accelerated math and eighth-grade algebra based on a defined prior achievement metric. This policy reduced the relationship between course assignment and student characteristics such as income and race/ethnicity, while increasing its relationship to academic skill. The policy increased the share of students on track for algebra by eighth grade. Students placed in accelerated math were exposed to higher-skilled peers but larger classes. Future work will assess impacts on subsequent achievement and course-taking outcomes.

Education agencies are evaluating teachers using student achievement data. However, very little is known about the comparability of test-based or “value-added” metrics across districts and the extent to which they capture variability in classroom practices. Drawing on data from four urban districts, we found that teachers were categorized differently when compared within versus across districts. In addition, analyses of scores from two observation instruments, as well as qualitative viewing of lesson videos, identified stark differences in instructional practices across districts among teachers who received similar within-district value-added rankings. These patterns were not explained by observable background characteristics of teachers, suggesting that factors beyond labor market sorting likely played a key role.

Background For nearly three decades, policy makers and researchers in the United States have promoted more intellectually rigorous standards for mathematics teaching and learning. Yet, to date, we have limited descriptive evidence on the extent to which reform-oriented instruction has been enacted at scale. Purpose The purpose of the study is to examine the prevalence of reform-aligned mathematics instructional practices in five U.S. school districts. We also seek to describe the range of instruction students experience by presenting case studies of teachers at high, medium, and low levels of reform alignment. Participants We draw on 1,735 video-recorded lessons from 329 elementary teachers in these five U.S. urban districts. Research Design We present descriptive analyses of lesson scores on a mathematics-focused classroom observation instrument. We also draw on interviews with district personnel, rater-written lesson summaries, and lesson video to develop case studies of instructional practice. Findings We find that teachers in our sample do use reform-aligned instructional practices, but they do so within the confines of traditional lesson formats. We also find that the implementation of these instructional practices varies in quality. Furthermore, the prevalence and strength of these practices corresponds to the coherence of district efforts at instructional reform. Conclusions Our findings suggest that, unlike other studies in which reform-oriented instruction rarely occurred, reform practices do appear to some degree in study classrooms. In addition, our analyses suggest that implementation of these reform practices corresponds to the strength and coherence of district efforts to change instruction.