Skills, Productivity and the Evaluation of Teacher Performance

Sass, Tim R.; Harris, Doug

doi:10.2139/ssrn.2020717

Cited by 5 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One indicator of this would be to examine whether the MGPs themselves align with other indicators of teacher quality derived, as also in this case, via the observations of the same teachers' practice at the same time using the TAP. Doing this is common, as well, across many other studies (see, for example, Goldring et al, 2015;Grossman et al, 2014;Hill et al, 2011;Kane & Staiger, 2012;Kersting et al, 2013; Measures of Effective Teaching (MET) Project, 2013; Polikoff & Porter, 2014;Rothstein & Mathis, 2013;Sass & Harris, 2012;Strunk et al, 2014); although, there are some researchers who disagree that these indicators should map onto the same construct given, for example, teaching is such a dynamic construct (see, for example, Braun, Goldschmidt, McCaffrey, Lissitz, 2012;Good, 2014;Harris, 2012;Kennedy, 2010;Martinez, Schweig, & Goldschmidt, 2016).…”

Section: Methodsmentioning

confidence: 90%

Not All Quartiles Are Equal: Evaluating Nonlinear Relationships Between Measures of Teacher Effectiveness

Pivovarova¹

2019

Proceedings of the 2019 AERA Annual Meeting

View full text Add to dashboard Cite

We estimated the relationship between two measures of teacher effectivenessmedian growth percentiles (MGPs) and observational scores. We did this for teachers' whose scores are at the different quartiles of the respective score distribution. The relationship between the two measures has been previously shown to be relatively weak, and we demonstrate that this weak correlation is even weaker for teachers in different quartiles. This weak correlation and the inconsistency between individual quartiles' and aggregate correlations can be partially explained by the non-linear relationship between MGPs and observational scores. This implies that when the two indicators of teacher quality are used together for evaluation purposes, teachers' positions in the score distribution should be considered in the design of any evaluation formula. ObjectivesThe primary objective of this study was to understand the relationship between the two most commonly used indicators of teacher effectivenessvalue-added, in this case as measured using teacher median growth percentiles (MGPs;Betebenner, 2009Betebenner, , 2016), 1 and TAP 2 observational scores. Existing evidence suggests that the correlation between multiple valueadded and observational measures of teacher quality is moderate or weak (Bill & Melinda Gates

show abstract

Section: Methodsmentioning

confidence: 90%

Not All Quartiles Are Equal: Evaluating Nonlinear Relationships Between Measures of Teacher Effectiveness

Pivovarova¹

2019

Proceedings of the 2019 AERA Annual Meeting

View full text Add to dashboard Cite

show abstract

“…Further, teachers can only truly assess their work when they have a clear understanding of the targets that their teaching practices are meant to achieve, and when two indicators of teacher quality produce conflicting results, the targets become even more blurred. This is an important issue to consider as states and districts try to follow recommendations of incorporating multiple measures of teacher quality, recommendations that are currently most popular among academics and researchers (see, for example, Harris, 2011;Hill et al, 2011;Kane & Staiger, 2012;Sass & Harris, 2012); though not enough has been done to this point to determine what level of correlations among multiple measures are appropriate enough to indicate validity. Meanwhile, it seems even two measures of teacher quality cannot be trusted to determine whether a teacher is "effective" or "ineffective," especially when one appears to influence or trump the value of the other.…”

Section: Validity Implicationsmentioning

confidence: 99%

Houston, We Have a Problem: Teachers Find No Value in the SAS Education Value-Added Assessment System (EVAAS®)

Collins

2014

EPAA

View full text Add to dashboard Cite

This study examined the SAS Education Value-Added Assessment System (EVAAS®) in practice, as perceived and experienced by teachers in the Southwest School District (SSD). To evaluate teacher effectiveness, SSD is using SAS EVAAS® for high-stakes consequences more than any other district or state in the country. A mixed-method design including a large-scale electronic survey was used to investigate the model's reliability and validity; to determine whether teachers used the SAS EVAAS® data in formative ways as intended; to gather teachers' opinions on SAS EVAAS®'s claimed benefits and statements; and to understand the unintended consequences that occurred as a result of SAS EVAAS® use in SSD. Results revealed that the reliability of the SAS EVAAS® model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the SAS EVAAS® results. The majority of teachers disagreed with the company's marketing claims and did not report similar SAS EVAAS® and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Many unintended consequences associated with the high-stakes use of SAS EVAAS® emerged through teachers' responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt epaa aape Education Policy Analysis Archives Vol. 22 No. 98 2 to raise SAS EVAAS® scores. The results of this study, one of the first to investigate how the SAS EVAAS® model works in practice, should be considered by policymakers, researchers, and districts when considering implementing the SAS EVAAS®, or any value-added model for teacher evaluation.Houston, tenemos un problema: Los docentes no le encuentran valor al sistema SAS de evaluación de valor agregado de la educación. Resumen: Este estudio examinó la implementación del sistema SAS de evaluación de valor agregado de la educación (EVAAS®), tal como fue percibido y experimentado por docentes en el Distrito Escolar del Suroeste (SSD). El SSD, está utilizando el SAS EVAAS® con consecuencias severas. más que cualquier otro distrito o estado en el país. Un diseño de métodos mixtos que incluye una encuesta electrónica a gran escala, se utilizó para investigar: la fiabilidad y la validez del modelo; para determinar si los profesores utilizan los datos SAS EVAAS® en maneras formativas según lo previsto; para reunir opiniones de los docentes sobre los beneficios y de acuerdo a lo que promociona la empresa que produce SAS EVAAS®; y para comprender las consecuencias no deseadas que se produjeron como consecuencia de la utilización SAS EVAAS® en SSD. Los resultados revelaron que la fiabilidad del modelo SAS EVAAS® produce resultados inconsistentes entre los participantes, y los profesores indicaron que los estudiantes sesgaban los resultados del SAS EVAAS®. La mayoría de los profesores no estuvo de acuerdo con los beneficios que el marketing de la empresa promocionaba y los ...

show abstract

“…Growth models are typically used for more descriptive purposes (Betebenner, 2011;Betebenner & Linn, 2010;Briggs & Betebenner, 2009;Linn, 2008) unless growth estimates correlate well with other measures of teacher effectiveness (Briggs & Betebenner, 2009). Researchers have found varying levels of correlations in their respective studies and are continuously discussing what size correlations might be appropriate or sufficient (Hill, Kapitula, & Umland, 2011;Kane & Staiger, 2012;Kersting, Chen, & Stigler, 2013;Sass & Harris, 2012). However, correlations are not inordinately high, and they are typically no greater than r = 0.50.…”

Section: The Policy Topographymentioning

confidence: 99%

Putting Growth and Value-Added Models on the Map: A National Overview

Collins

Amrein‐Beardsley

2014

Teachers College Record: The Voice of Scholarship in Education

View full text Add to dashboard Cite

Background Within the last few years, the focus on educational accountability has shifted from holding students responsible for their own performance to holding those shown to impact student performance responsible—students’ teachers. Encouraged and financially incentivized by federal programs, states are becoming ever more reliant on statistical models used to measure students’ growth or value added and are attributing such growth (or decline) to students’ teachers of record. As states continue to join the growth and value-added model movement, it is difficult to find inclusive resources documenting the types of models used and plans for each state. Objective To capture state initiatives in this area, researchers collected data from all 50 states and the District of Columbia to provide others with an inclusive national growth and value-added model overview. Data yielded include information about the types of growth or value-added models used in each state, the legislature behind each state's reform efforts, the standardized tests used for growth or value-added calculations, and the strengths and weaknesses of each state's models as described by state personnel. Method This article synthesizes qualitative and quantitative themes as identified from data collected via multiple phone interviews and emails with state department of education personnel in charge of their own state's initiatives in this area, as well as state websites. These data provide the most inclusive and up-to-date resource on national growth and value-added data usage, noting however that this is changing rapidly across the nation, given adjustments in policies, pieces of legislation, and the like. Conclusions Findings from this study provide a one-stop resource on what each state has in place or in development regarding growth or value-added model use as a key component of its state-based teacher evaluation systems. Despite widespread use, however, not one state has yet articulated a plan for formative data use by teachers. Federal and state leaders seem to assume that implementing growth and value-added models leads to simultaneous data use by teachers. In addition, state representatives expressed concern that the current emphasis on growth and value-added models could be applied to only math and English/language arts teachers with state standardized assessments (approximately 30% of all teachers). While some believe the implementation of the Common Core State Standards and its associated tests will help to alleviate such issues with fairness, more research is needed surrounding (the lack of) fairness and formative use associated with growth and value-added models.

show abstract

Skills, Productivity and the Evaluation of Teacher Performance

Cited by 5 publications

References 39 publications

Not All Quartiles Are Equal: Evaluating Nonlinear Relationships Between Measures of Teacher Effectiveness

Not All Quartiles Are Equal: Evaluating Nonlinear Relationships Between Measures of Teacher Effectiveness

Houston, We Have a Problem: Teachers Find No Value in the SAS Education Value-Added Assessment System (EVAAS®)

Putting Growth and Value-Added Models on the Map: A National Overview

Contact Info

Product

Resources

About