An Evaluation of Empirical Bayes’s Estimation of Value-Added Teacher Performance Measures

Abstract

Empirical Bayes's (EB) estimation has become a popular procedure used to calculate teacher value added, often as a way to make imprecise estimates more reliable. In this article, we review the theory of EB estimation and use simulated and real student achievement data to study the ability of EB estimators to properly rank teachers. We compare the performance of EB estimators with that of other widely used value-added estimators under different teacher assignment scenarios. We find that, although EB estimators generally perform well under random assignment (RA) of teachers to classrooms, their performance suffers under nonrandom teacher assignment. Under non-RA, estimators that explicitly (if imperfectly) control for the teacher assignment mechanism perform the best out of all the estimators we examine. We also find that shrinking the estimates, as in EB estimation, does not itself substantially boost performance.

Empirical Bayesball Remixed: Empirical Bayes Methods for Longitudinal Data

“…One prominent source of such applications is the literature on teacher evaluation. Guarino et al (2015) have recently argued that empirical Bayes methods may be misguided when teacher assignment is closely tied to student performance but, as expected, they show that these methods perform well under random assignment. Prediction of insurance liability claims also relies heavily on the linear shrinkage rules arising from the Gaussian random effects paradigm as demonstrated in Bühlmann and Gisler (2005) and the extensive related literature in actuarial science.…”

Teacher and Teaching Effects on Students’ Attitudes and Behaviors

“… 9 An alternative approach would be to specify teacher effects as fixed, rather than random, which relaxes the assumption that teacher assignment is uncorrelated with factors that also predict student outcomes (Guarino, Maxfield, Reckase, Thompson, & Wooldridge, 2015). Ultimately, we prefer the random effects specification for three reasons.…”

Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels

“…This differs from other papers on shrinkage, which examine how shrinkage affects teacher ranks (Tate 2004;Guarino et al 2012), and other work on heteroskedasticity, which examines its effect on the precision and inter-year stability of valueadded estimates (Stacy et al 2012). Our purpose is to examine how shrinkage affects the probability that teachers of hard-to-predict students are classified at the extremes of the value-added distribution of teacher effectiveness, because many evaluation systems use these thresholds to determine consequences.…”

“…The adjusted 1 A number of studies have examined the extent to which nonrandom assignment of students to teachers causes bias in teacher value-added estimates and teachers to be misclassified (for example, Aaronson et al 2007; Kane and Staiger 2008;Rothstein 2010;Chetty et al 2011;Guarino et al 2012). Although these studies tend to focus on the point estimates, this paper examines the estimates' variances.…”