Measuring Teaching Using Value-Added Modeling

Abstract

The use of value-added modeling (VAM) in school accountability is expanding. However, trying to decide how to embrace VAM can be rather nettlesome. Some experts claim it is "too unreliable," causes "more harm than good," and has "a big margin for error," while other experts assert VAM is "imperfect, but useful" and provides "valuable feedback." This article attempts to parse these statements by exploring the underlying statistical assumptions of VAM, the reliability of VAM's estimates, and the validity of the inferences commonly made based on the estimates of VAM. It then goes on to discuss the perverse incentives, unintended consequences, and gaming that might accompany the misuse of VAM. The article concludes that while, in many cases, VAM may be preferable to other commonly used measurement modes, it should never be used as the sole indicator of teacher effectiveness. Rather, it should just be a piece of a larger accountability system.

Educational effectiveness in Chilean secondary education: comparing different ‘value added’ approaches to evaluate schools

“…In the international scenario there is considerable literature from western countries in general (Willms & Raudenbush, 1989;Raudenbush & Willms, 1995;Ladd & Zelli, 2002;Hoyle & Robinson, 2003;OECD, 2008OECD, , 2011aRosenkvist, 2010;Scherrer, 2011) and the UK in particular (Goldstein et al, 1993;Sammons et al, 1997;Slee, Weiner & Tomlinson, 1998;Sammons, 1999;De Luca, in Lambert & Lines, 2000;Leckie & Goldstein, 2007;Leckie, 2008;Thomas, Salim, Muñoz-Chereau, & Peng, 2012) supporting the argument that raw or unadjusted measures of pupil achievement in a given examination or test provide a poor method for comparing schools' performance. Particularly in Chile, the decision to judge school performance based on their raw test results would be a way of rewarding social segregation (Treviño & Donoso, 2010).…”

“…The claim 1 While the average percentage of total variance explained by socioeconomic background factors across OECD countries is 14%, in Chile the equivalent figure is 57% (OECD, 2010). that something more sophisticated than raw 'league tables' is needed in order to compare schools on a more equitable basis has been strongly associated with VA and CVA approaches (Schagen & Hutchison, 2003), because they offer the promise of a more rigorous approach for levelling the playing field (Braun, Chudowsky & Koenig, 2010;Scherrer, 2011).…”

Value-Added Model (VAM) Research for Educational Policy: Framing the Issue.

“…Yet while a growing volume of published research has revealed substantive methodological and inferential concerns when evaluating teacher effectiveness using VAM approaches (see, for example, Amrein-Beardsley, 2008;Corcoran, 2010;Darling-Hammond, Amrein-Beardsley, Haertel, & Rothstein, 2012;Muijs, 2006;Papay, 2012;Raudenbush, 2004;Rivkin, 2007;Rothstein, 2009;Rubin, Stuart, & Zanutto, 2004;Scherrer, 2011;Schochet & Chiang, 2010;Zeis, Waronska, & Fuller, 2009), this dialogue is unfortunately taking place among academic researchers and not between academic researchers and policymakers. The translation of research to practice and policy development has yet to be realized in significant and impactful ways.…”

The Negative Effects of Student Mobility: Mobility as a Predictor, Mobility as a Mediator

“…Mobile students often have an "incomplete data set" and are not accounted for in such systems. By not including mobile students in accountability measures, teachers have the perverse incentive to give a disproportionate amount of energy to the students who will "count" (Scherrer, 2011).…”