The Limitations of Teacher Pay Incentive Programs Based on Inter-Cohort Comparisons: The Case of Chile's SNED

Abstract

Since 1996, the Chilean government has awarded teachers pay bonuses based on school performance using a complex formula that combines absolute average student test scores and inter-cohort gains from test year to test year. In this paper, we compared the bonuses schools actually received on the basis of this formula to how they would have fared under a hypothetical alternative measure of school performance—intra-cohort gains between the 4th and 8th grades in 1996–2000. We show that schools that received monetary premiums for “good performance” under the SNED program were more likely to be schools that had scored higher on the 4th grade 1996 test, but, on average, they were not the schools that made the highest cohort gains as students progressed from 4th grade in 1996 to 8th grade in 2000. Given what we have found, to get more SNED awards, the wise school would do much better to raise 4th grade and 8th grade scores every two years (even years for 4th grade and odd years for 8th grade) and not focus on the more difficult task of helping students make greater progress from 4th to 8th grades. Although we have data on only one cohort's gain scores from 4th to 8th grades, our limited analysis demonstrates the possible constraints of a school-based incentive program when the available student performance data for awarding pay bonuses are inter-cohort rather than intra-cohort test score gains.

In Search of Good Teachers: Patterns of Teacher Quality in Two Mexican States

“…One common element of these efforts is the use of incentives to recruit and retain qualified teachers, as well as to encourage improved performance among practicing teachers (OECD 2005). For example, Chile's Sistema Nacional de Evaluación del Desempeño de los Establecimientos Educacionales Subvencionados (SNED) provides salary bonuses to teachers based on schoolwide achievement gains (Carnoy et al 2007). In the United States, the National Commission on Teaching and America's Future (NCTAF) issued an influential report in 1996 calling for a greater focus on teacher quality and training, including a call to strengthen standards and incentives for classroom teachers.…”

“…Teacher professionalization efforts in other countries also include incentives to raise student achievement. Chile's SNED provides schoolwide bonuses to teachers in schools that increase student achievement scores from year to year (Carnoy et al 2007). Mexico's CM program provides significant salary bonuses to teachers based on their own training and experience and student achievement scores.…”

“…Mexico's CM program provides significant salary bonuses to teachers based on their own training and experience and student achievement scores. To date, evaluations of these programs have found little evidence that they have led to improved student outcomes (McEwan and Santibañez 2005;Carnoy et al 2007;Harris and Sass 2009).…”

Handing Over the School Keys: The Impact of Privatisation on Education Quality

“…It was only in the 1990s that school results were published, but these data do not separate pupil ability from school effectiveness. The school effectiveness measures that have been published have also suffered from poor construction, and parents have not used them when choosing schools (Carnoy et al ; Mizala and Urquiola ). Poor information is likely to have spurred parents to choose schools mainly on the basis of peer characteristics instead of school quality; and it is unlikely that such sorting has large net effects, either positive or negative, on achievement (Sacerdote ).…”

Chile: Climbing on Giants' Shoulders

“…Until recently the SIMCE test had not allowed calculating intra-cohort gains,a measure of value added -because the same class was never tested twice. Nevertheless, Carnoy et al (2007b) were able to show that schools that received the SNED on average were not likely to be the ones that achieved the highest intra-cohort gains between 1996 and 2000. This suggests that it might be preferable to base performance-pay on richer information than school rankings of SIMCE scores adjusted for socio-economic background.…”