Linking Student Outcomes to School Administrator Discretion in the Implementation of Teacher Observations

Abstract

Purpose: Many studies have explored school administrator discretion in the implementation of teacher evaluation and observation systems. However, we are unaware of any studies that quantitatively link discretionary administrator behaviors to student outcomes. The purpose of this study was to (a) explore the determinants of observations arising from administrator discretion and (b) explore the extent to which “discretionary observations” were associated with average student achievement scores and disciplinary offenses. Method: We applied multilevel modeling to 3 years of teacher panel data from more than 80% of Tennessee school districts. Findings: Observable characteristics, differences between schools each year, and teacher traits explain more than 80% of the variation in discretionary observations; teacher prior-year observation and composite effectiveness scores were the strongest predictors. No evidence suggested that average student achievement scores or behavior suffered among teachers who received fewer observations than assigned by policy. Average student achievement scores rose among teachers receiving supplementary observations compared with the years when they received the policy-prescribed number. Implications for Research and Practice: Quantitative research can substantially control for discretionary observations using multilevel modeling. Observers might be encouraged to not observe high-performing teachers more than what is prescribed by policy. Observers seemingly deviate from observation policy in ways that do not increase office referrals and may improve student achievement.

Critical Feedback Characteristics, Teacher Human Capital, and Early-Career Teacher Performance: A Mixed-Methods Analysis

“…1 A growing body of work suggests that the success of these reforms rests partly on teacher observation processes and the policies that shape them (Donaldson, 2021). Specifically, teaching improvements seem to depend on the quality of observers’ training (Steinberg & Sartain, 2015), observers’ discretion in the implementation of teacher observations, structured observation conferences, or post-observation improvement plans (Donaldson & Woulfin, 2018; Hunter & Ege, 2021; Marsh et al, 2017), and teachers’ receipt of observations (Phipps, 2018; Phipps & Wiseman, 2021). Recent quasi-experimental work also indirectly implies that observations improve teachers’ performance by increasing their motivation via accountability mechanisms and performance feedback (Phipps & Wiseman, 2021).…”

Explaining variation in the implementation of observational processes by school leaders: evidence from a Tennessee-based researcher–practitioner partnership

“…Although a previous study concluded that school administrators deliver higher dosages of effective feedback to historically less effective early-career teachers (Hunter and Springer, 2022), findings from the current study and other prior research suggest the opposite. For example, Hunter and Ege (2021) studied school administrator implementation of teacher observation processes in a setting where highly effective teachers are assigned one or two observations, and the least effective teachers are assigned four. In that study, school administrators visited the classrooms of teachers needing the least improvement (i.e.…”

“…In that study, school administrators visited the classrooms of teachers needing the least improvement (i.e. historically highly effective teachers) more than the number of times assigned by policy (Hunter and Ege, 2021). The current study also finds that administrators implement observations in ways that may not improve teaching among those needing it the most by providing historically less effective teachers with less well-implemented observations [7].…”

“…A nascent body of work finds that observable teacher, school administrator and contextual characteristics predict some aspects of observation implementation (Hunter and Ege, 2021). The italicized terms in this paragraph refer to the characteristics measured by the study herein (see italicized text in Figure 1).…”

“…A nascent body of work finds that observable teacher, school administrator and contextual characteristics predict some aspects of observation implementation (Hunter and Ege, 2021).…”