Pitfalls in research on school and teacher effectiveness

Abstract

Many pitfalls in research on school and teacher effectiveness are described. The aspects of effectiveness research considered include the relationship between correlation and causation, statistical significance, practical significance, instrumentation problems, controlling for background factors, regression effects, the measurement of change, non-linear relationships, aptitude-treatment interactions and levels of analysis. Despite the many pitfalls in effectiveness research, one conclusion does emerge: differential school effects are relatively modest and unlikely to be clearly related to the national assessment test averages that schools will be required to publish.

Short‐ and Long‐term Follow‐up of Parental Involvement in Reading Projects

“…The use of change scores has been criticised by Plewis (1985) for example, who prefers a regression analysis approach. On the other hand, Preece (1989) notes that the use of change scores has been vigorously defended, and that regression analysis may suffer from bias in this application when subject to the vagaries of opportunistically gathered field data.…”

Estimating Differences in the Examination Performances of Secondary Schools in Six LEAs: a multi‐level approach to school effectiveness

“…Crucially, each of the studies mentioned above has produced somewhat differing estimates of schools' effectiveness because each has, in practice, conceptualised the problems somewhat differently and pursued them using data collected from different localities incorporating different variables. An extended review of some of Downloaded by [Queensland University of Technology] at 17:09 19 October 2014 these issues, as well as some more general issues raised by studies of school and teacher effectiveness, has recently been completed by Preece (1989).…”

The classroom competence and attitudes towards pedagogical principles of beginning teachers

“…It seems to be the case that more direct, didactic, teaching styles, for example, which directly promote greater achievement also give rise to more negative attitudes towards teachers (Peterson, 1979). Furthermore, a re-analysis of National Assessment of Educational Progress data relating to the science achievement of 17-year-olds indicated a direct positive causal path between the didacticism of teaching and pupil learning but a negative indirect causal path between didacticism and learning via class morale (Preece, 1989). Thus if class morale were held constant, more didactic teaching would lead to greater pupil achievement, but in practice a more didactic approach generally leads to more negative attitudes and hence to lower achievement.…”