Building a More Complete Understanding of Teacher Evaluation Using Classroom Observations

Abstract

Improving teacher evaluation is one of the most pressing but also contested areas of educational policy. Value-added measures have received much of the attention in new evaluation systems, but they can only be used to evaluate a fraction of teachers. Classroom observations are almost universally used to assess teachers, yet their statistical properties have received far less empirical scrutiny, in particular in consequential evaluation systems. In this essay, we highlight some conceptual and empirical challenges that are similar across these different measures of teacher quality. Based on a review of empirical research, we argue that we need much more research focused on observations as performance measures. We conclude by sketching out an agenda for future research in this area.Keywords: accountability; classroom research; educational policy; policy analysis; teacher assessment REVIEWS/ESSAySAUgUST/SEpTEMBER 2016 379Interestingly, given their prevalent use, we know surprisingly little about the statistical properties of classroom observations in consequential personnel decisions. Indeed, much of what we know is derived from extensive research of a large-scale research study-the Measures of Effective Teaching (MET) Project (cf. Kane & Staiger, 2012)-and it is unclear how these findings might translate when evaluation reform is put into practice (Goldhaber, 2015). Will real-world classroom observations differentiate among teachers? Will they be reliable? Will teachers receive actionable feedback, leading them to seek and receive high-quality professional development? The answers to questions like these are key to understanding how the use of observational measures of performance will affect the quality of the teacher workforce.In this article, we highlight some conceptual and empirical challenges that are similar across different measures of teacher performance. We focus on the existing validity evidence around classroom observations as evaluation measures, much of which is derived from research studies as opposed to real-world evaluation systems. We highlight what we know about the stability of observational measures across raters and educational contexts. We speculate on the implications of the extant literature, and we ask what additional kinds of evidence we would need about observations to feel confident observations could be featured in fair and useful evaluation systems. We attempt to answer some of these questions while describing the conceptual issues that arise when measuring teachers' classroom performance based on observations. Based on a review of empirical research, we argue that we need more research focused on observations as performance measures, particularly from authentic settings where observational performance measures are used in consequential evaluation systems, and we sketch out an agenda for future research. Given space constraints, our goal is not to provide a comprehensive synthesis of the extant literature around classroom observations. Such a review would be helpful but goes beyond th...

Teacher Skill Development: Evidence from Performance Ratings by Principals

“…Beneath the turbulent surface of these contentious reforms, the longstanding practice of administrators evaluating teachers based on classroom observations has remained the dominant component of the evaluation process (Steinberg & Donaldson, 2016). The resilience and pervasiveness of this practice makes understanding what ratings capture and what we can learn from subjective evaluation ratings a continued priority (Cohen & Goldhaber, 2016).…”

How middle school special and general educators make sense of and respond to changes in teacher evaluation policy

“…The use of classroom observations to evaluate teachers is common practice, yet there is much inconsistency in the focus, duration, and frequency of classroom observations (Pianta & Hamre, 2009). Moreover, the observational instruments that are used in practice are rarely consistent across states, districts, or schools (Cohen & Goldhaber, 2016). To ensure evidence-based teacher evaluation policy and practice, it is important that principals and school districts use classroom observation tools that have demonstrated reliability and validity as part of their teacher evaluation process (Marx, 2014;Pianta, 2012).…”

Examining the Factor Structure Underlying the TAP System for Teacher and Student Advancement

“…Kane, Kerr, & Pianta, 2014;Steinberg & Garrett, 2016). However, the intent has historically been formative, providing teachers with targeted feedback to improve pedagogical competency (Cohen & Goldberger, 2016;Danielson, 2010Danielson, , 2011Steinberg & Garrett, 2016). Not until recently has this focus evolved in response to the same high-stakes policy-based accountability reforms.…”

“…Notwithstanding, there remains much controversy over the appropriateness of these test-base metrics as valid representations of teachers' instructional competencies and effects, especially when consequential decisions (e.g., teacher merit pay, tenure, termination) are to be attached to such measures (Baker et al, 2010;Berliner, 2005;Cohen & Goldberger, 2016;DarlingHammond, Amrein-Beardsley, Haertel, & Rothstein, 2012;Papay, 2011).…”

“…Traditionally, school practitioners have enjoyed widespread use of observation-based evaluation systems to examine teachers' instructional practice (Blank, 2010;Cohen & Goldberger, 2016; T. J. Kane, Kerr, & Pianta, 2014;Steinberg & Garrett, 2016).…”