Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement

Abstract

Growing concerns over the achievement of U.S. students have led to proposals to reward good teachers and penalize (or fire) bad ones. The leading method for assessing teacher quality is "value added" modeling (VAM), which decomposes students' test scores into components attributed to student heterogeneity and to teacher quality. Implicit in the VAM approach are strong assumptions about the nature of the educational production function and the assignment of students to classrooms. In this paper, I develop falsification tests for three widely used VAM specifications, based on the idea that future teachers cannot influence students' past achievement. In data from North Carolina, each of the VAMs' exclusion restrictions are dramatically violated. In particular, these models indicate large "effects" of 5th grade teachers on 4th grade test score gains. I also find that conventional measures of individual teachers' value added fade out very quickly and are at best weakly related to long-run effects.

Class-size reduction policies and the quality of entering teachers

“…Viewed in this light, the challenges of VAM estimation are those faced in identifying causal relationships with panel data more generally. VAM estimation has proven to be difficult in non-experimental settings and there is no consensus on what the best model of student achievement is or the best approach to estimating the portion attributable to teachers (McCaffrey et al 2004;Kane & Staiger 2008, Rothstein 2009, 2010Koedel & Betts 2011). Much of this difficulty stems from the non-random assignment of students to teachers both within and across schools.…”

“…For instance, if a principal reacts to a randomly good or bad test score in one year when determining a future teacher assignment, this would violate strict exogeneity. As noted by Rothstein (2009Rothstein ( , 2010, the fixed effects approach is useful when assignment to teachers is made based on a static characteristic of the student. The usefulness of FE estimation breaks down some when assignment decisions are made dynamically based on new information gathered over time by the relevant decision makers, be it principals, parents, or the students.…”

Do Typical RCTs of Education Interventions Have Sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes?

“…Importantly, the classroom-level error θ ij y reflects classroom-level variation in student test score gains, including both persistent and transitory effects (such as a "barking dog" effect that influences all students in the classroom at the time the test is administered). The literature provides separate estimates for these two effects using longitudinal student and teacher data and estimating models similar to (1) (see, for example, Goldhaber 2002;Hanushek et al 2005;Jacob and Lefgren 2005;McCaffrey et al 2004;Nye et al 2004;and Rothstein 2009). Estimates of the persistent classroom effects may capture teacher-, student-, and school-related factors that influence student achievement.…”

Evaluating Specification Tests in the Context of Value-Added Estimation

“…Given the several choices among estimators in the VAM context it would be helpful to have tools for choosing among different estimators, especially when they produce very different estimated effects or, more fundamentally, whether any estimator does a good enough job of estimating effects to enable the use of these estimates for policy purposes, such as rewarding or sanctioning teachers based on estimated performance. It is logical to turn to the large array of statistical tests that currently exist to help diagnose whether or not underlying assumptions are met; many have been developed in the statistical and econometric literature in reference to other topics and issues, and some recent tests have been proposed by education researchers for the express purpose of evaluating VAMs (for example, those in Rothstein (2010) and Harris, Sass, and Semykina (2010)). …”

“…Rothstein (2010) and Harris, Sass, and Semykina (2010) are two examples of studies that develop and apply statistical tests of assumptions to VAMs designed to produce measures of teacher effectiveness.…”