Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

Goldhaber, Dan; Walch, Joe; Gabele, Brian

doi:10.1080/2330443x.2013.856169

Cited by 59 publications

(65 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, locales from Washington, DC, to New York City to the state of Missouri are experimenting with VAM-based accountability in a variety of forms. 2 Although the research literature on growth modeling is vast, most studies do not illustrate the tradeoffs that come with choosing different VAM specifications (a recent exception is Goldhaber et al 2012). The VAM literature is too large to cite individual studies but a useful starting point for the interested reader is crodata from Missouri, we compare models that differ in terms of the number of lagged test scores and the types of other control variables that are included.…”

Section: Introductionmentioning

confidence: 97%

The Sensitivity of Value-Added Estimates to Specification Adjustments: Evidence From School- and Teacher-Level Models in Missouri

Ehlert¹,

Koedel²,

Parsons³

et al. 2013

Statistics and Public Policy

View full text Add to dashboard Cite

We provide a side-by-side comparison of school and teacher growth measures estimated from different value-added models (VAMs). We compare VAMs that differ in terms of which student and school-level (or teacher-level) control variables are included and how these controls are included. Our richest specification includes 3 years of prior test scores for students and the standard demographic controls; our sparsest specification conditions only on a single prior test score. For both schools and teachers, the correlations between VAM estimates across the different models are high by conventional standards (typically at or above 0.90). However, despite the high correlations overall, we show that the choice of which controls to include in VAMs, and how to include them, meaningfully influences school and teacher rankings based on model output. Models that are less aggressive in controlling for student-background and schooling-environment information systematically assign higher rankings to more-advantaged schools, and to individuals who teach at these schools.

show abstract

Section: Introductionmentioning

confidence: 97%

The Sensitivity of Value-Added Estimates to Specification Adjustments: Evidence From School- and Teacher-Level Models in Missouri

Ehlert¹,

Koedel²,

Parsons³

et al. 2013

Statistics and Public Policy

View full text Add to dashboard Cite

show abstract

“…Thus, I also use a continuous measure of model agreement by examining the correlations between the estimated teacher effects across models. This methodology has also been used in prior work (Tekwe et al 2004;Goldschmidt, Choi, and Beaudoin 2012;Ehlert et al 2013;Goldhaber, Walch, and Gabele 2014).…”

Section: Resultsmentioning

confidence: 99%

“…Both VAM and SGP models are capable of a large degree of complexity (Betebenner 2009;Wright 2010;Goldhaber, Walch, and Gabele 2014). However, for comparison purposes, the models examined herein have been simplified to their most basic levels.…”

Section: Methodsmentioning

confidence: 99%

“…These data allow for the linking of student records regarding test scores and demographics to teachers, and offer the ability to track particular teachers over time. These data have been used extensively in prior research on related topics, largely because of the ability to link teachers and students (Clotfelter, Ladd, and Vigdor 2006;Goldhaber, Walch, and Gabele 2014). I restrict my sample to 4th grade math teachers linked to 4th grade end-of-grade math exams because these students are most likely to have a welldefined general or math-specific teacher.…”

Section: Datamentioning

confidence: 99%

See 1 more Smart Citation

Value-Added and Student Growth Percentile Models: What Drives Differences in Estimated Classroom Effects?

Kurtz

2018

Statistics and Public Policy

View full text Add to dashboard Cite

This study shows value-added models (VAM) and student growth percentile (SGP) models fundamentally disagree regarding estimated teacher effectiveness when the classroom distribution of test scores conditional on prior achievement is skewed (i.e., when a teacher serves a disproportionate number of high-or low-growth students). While conceptually similar, the two models differ in estimation method which can lead to sizable differences in estimated teacher effects. Moreover, the magnitude of conditional skewness needed to drive VAM and SGP models apart often by three and up to 6 deciles is within the ranges observed in actual data. The same teacher may appear weak using one model and strong with the other. Using a simulation, I evaluate the relationship under controllable conditions. I then verify that the results persist in observed student-teacher data from North Carolina.

show abstract

“…The VA scores are highly (but not perfectly) correlated across the different models, but, as Goldhaber, Walch, and Gabele (2014) demonstrated, those high correlations do not prevent some teachers from being ranked differently by different models. The models all produce estimates that are on different scales; to compare them, Table 1 displays the percentile ranks for Teachers A and B from the different models.…”

Section: Persistence Modelsmentioning

confidence: 96%

Red Beads and Profound Knowledge: Deming and Quality of Education

Lohr

2015

EPAA

View full text Add to dashboard Cite

Citation: Lohr, S. (2015). Red beads and profound knowledge: Deming and quality of education. Education Policy Analysis Archives, 23(80). http://dx.doi.org/10. 14507/epaa.v23.1974 Abstract: Value-added models are being implemented in many states in an attempt to measure the contributions of individual teachers and schools toward students' learning. Scores from these models are increasingly used for high-stakes purposes such as setting compensation, hiring or dismissing teachers, awarding tenure, and closing schools. The statistician W. Edwards Deming wrote extensively about improving quality in education and the damage caused by performance rankings. We examine uses and misuses of value-added models in the context of Deming's System of Profound Knowledge, and discuss contributions a Deming-based perspective and statistical science can make to improving education. Keywords: educational improvement; educational quality; sampling; statistics; systems approach; teacher evaluation Cuentas Rojas y Conocimientos Profundos: La Perspectiva de W.E. Deming y la Calidad de la Educación Resumen: Los modelos de valor añadido se están implementando en muchos estados, en un intento de medir las contribuciones individuales de los profesores y escuelas en el aprendizaje de los estudiantes. Las puntuaciones de estos modelos se utilizan cada vez más para fines de consecuencias severa, tales como el establecimiento de una indemnización, contratación o despido de docentes, la concesión de una escuela a una firma, o el cierre de las escuelas. El estadístico W. Edwards Deming escribió extensamente acerca de la mejora de la calidad en la educación y los daños causados por los epaa aape Education Policy Analysis Archives Vol. 23 No. 80 2 rankings de desempeño. Examinamos usos y abusos de los modelos de valor agregado en el contexto del Sistema de Conocimiento Profundo de Deming, y discutimos las contribuciones de una perspectiva basada en Deming y como la estadística pueden contribuir para mejorar la educación. Palabras clave: mejora educativa; calidad de la educación; muestreo; estadísticas; sistemas; evaluación docentesContas Vermelhas e Conhecimentos Profundos: A Perspectiva de W.E. Deming e a Qualidade da Educação Resumo: Os modelos de valor agregado estão sendo implementadas em muitos estados, em uma tentativa de medir as contribuições individuais dos professores e das escolas no aprendizagem dos alunos. Dezenas de estes modelos são utilizados cada vez mais para fins de consequências severas, como a determinação da remuneração, contratação e demissão de professores, administração escolar cedidas a empresas, ou o fechamento de escolas. Estatístico W. Edwards Deming escreveu extensivamente sobre como melhorar a qualidade da educação e sobre os danos causados pelos rankings de desempenho. Nós examinamos o uso e abuso de modelos de valor agregado no contexto do Sistema de Deming do conhecimento profundo, e discutimos as contribuições de uma perspectiva baseada em Deming e como as estatísticas podem contribuir para melhorar a educação.

show abstract

Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

Cited by 59 publications

References 43 publications

The Sensitivity of Value-Added Estimates to Specification Adjustments: Evidence From School- and Teacher-Level Models in Missouri

The Sensitivity of Value-Added Estimates to Specification Adjustments: Evidence From School- and Teacher-Level Models in Missouri

Value-Added and Student Growth Percentile Models: What Drives Differences in Estimated Classroom Effects?

Red Beads and Profound Knowledge: Deming and Quality of Education

Contact Info

Product

Resources

About