Stability of School-Building Accountability Scores and Gains

Abstract

A number of states have school-building accountability systems that rely on comparisons of achievement from one year to the next. Improvement of the performance of schools is judged by changes in the achievement of successive groups of students. Year-to-year changes in scores for successive groups of students have a great deal of volatility. The uncertainty in the scores is the result of measurement and sampling error and nonpersistent factors that affect scores in one year but not the next. The level of uncertainty was investigated using fourth grade reading results for schools in Colorado for four years of administration of the Colorado Student Assessment Program. It was found that the year-to-year changes are quite unstable, resulting in a near zero correlation of the school gains from years one to two with those from years three to four. Some suggestions for minimizing volatility in change indices for schools are provided.

The Impact of a Professional Development Program on Student Achievement in Grade 6 Mathematics

“…The best argument for the claim that the evaluation was premature was the change in student achievement on the provincial assessments written in May 2004: scores increased significantly from 2003 to 2004, although the effect size was small. Given the finding of Linn and Haug (2002) that external assessment results for individual schools are highly unstable from one year to the next, it could be argued that the differences between 2003 and 2004 achievement in our study represent changes in the ability of the two cohorts and/or changes in the difficulty of the tests. However, there were over 3000 students involved in the assessments in both years, diluting the effects of between-cohort fluctuations.…”

Variability in Percentage Above Cut Scores Due to Discreteness in Score Scale

“…Researchers have conducted various studies to evaluate the degree that the irrelevant factors affect classification rates and hence accuracy in the performance trend. Arce‐Ferrer, Frisbie, and Kolen (), Linn and Haug (), Kane and Staiger (), and Yen () pointed to sampling error as a significant factor accounting for variability in achievement‐level percentages. Betebenner, Shang, Xiang, Zhao, and Yue () quantified the extent to which measurement error produced bias and increased error variability for percentage at performance‐level statistics.…”

Do increased resources increase educational attainment during a period of rising expenditure? Evidence from English secondary schools using a dynamic panel analysis

“…total examination performance scores) over time (correlations are fairly strong and all positive)’. Other authors, however, report greater instability when cohorts stretch over longer periods and cover several years (see, for example, Gray, 2001; Linn & Haug, 2002). Using dynamic panel data analysis over a longer time scale, Mangan et al (2005) found considerable variation in school improvement paths and a degree of persistence with respect to performance at school level, with significant two‐year lags.…”