A note on national assessment and school comparisons

A Brief History of Educational 'Value Added': How Did We Get To Where We Are?

“…If these factors were not taken into account, the performance of schools could be neither fairly assessed nor validly compared, as TGAT themselves intimated. But, according to Goldstein and Cuttance (1988), TGAT could be accused of wanting it both ways: on the one hand, failing actually to mention the school effectiveness research so far published whilst on the other -by recommending that local socio-economic (SE) factors should be published 'alongside' schools' results -passing the buck to parents and LEAs for making the necessary adjustments to schools' results in order to get a truer picture of their effectiveness. The authors conclude (p. 201) that:…”

“…Finally, a point made by both Smith and Tomlinson (1989) and Goldstein and Cuttance (1988) (and which can be easily inferred from Nuttall et al, 1989) is one that follows inexorably from the ability to make fairer and more valid comparisons between schools on the basis of pupils' progress, but which tends to be obscured by the term 'value added'. Smith and Tomlinson (1989) put it like this: 'A school having a low balance of intake could appear to be doing badly, when in fact it was doing well; while a school with a high intake balance could be flattered by the raw test results' (p. 303).…”

“…Before the end of that same year a short paper by Goldstein and Cuttance (1988) was published in which the authors explicitly forewarned of, and tried to forestall, what they called the 'handicapping system ' (p. 200) which would in all likelihood do an injustice both to schools as would-be providers and to parents as would-be selectors of educational quality. The authors argued that if the proposed system were to make comparisons between schools on the basis of pupils' attainment in circumstances where there was variation, either social and/or academic, in pupil intakes to schools -and it could be assumed that in the real world there would normally be such variationthis would mask the true extent of the progress made by pupils in different schools, as distinct from the standards reached by them.…”

“…Presciently, Goldstein and Cuttance (1988) introduce their paper by warning that 'publication [of comparative school results] could well become the most visible and perhaps most important aspect of the whole system, irrespective of the actual content of the assessment or indeed of the national Curriculum itself. ' (p. 197) On more substantive matters, Goldstein and Cuttance point out that there are flaws in the system at a basic level, in that a comparison of school averages: tells us nothing about the relative achievements of different types of pupils within the schools… Consequently schools which perform well relative to other schools for the average pupil in the population may perform less well for disadvantaged or advantaged pupils.…”

Pitfalls in research on school and teacher effectiveness

“…Indeed, the question of which background characteristics should be taken into account was one of the issues on which there was no general agreement at the first International Conference on Effective Schools (Goldstein and Cuttance, 1988). Woodhouse and Goldstein (1988) have noted, for example, how three independent analyses of the same aggregate LEA data, each controlling for a different set of input variables, resulted in markedly different rankings for some LE As.…”