Methodological Issues in Educational Performance and School Effectiveness Research: A Discussion with Worked Examples

Abstract

There has been a growing awareness among educational researchers of the consequences of using data-analytic models that fail to account for the inherent clustered or hierarchical sampling structure of the data typically obtained. Such clustering poses special analytic problems related to levels of analysis, aggregation bias, heterogeneity of regression and parameter mis-estimation, with important implications for the correct interpretation of effects. This paper compares the results obtained from fitting single-level and multilevel models to two hierarchically structured data sets designed to explain variation in student achievement. Emphasis is given to the crucial importance of fitting models commensurate with the sampling structure of the data to which they are applied.

Undergraduates’ learning experience and learning process: quantitative evidence from the East

“…After the confirmatory and validation analysis on six-one-factor congeneric models (four SCEQ and two SPQ models), a composite score was calculated for each of the six scales based on factor score regression weights produced in the LISREL output estimates using a non-unit weighted score which reflected the actual contribution each item made to the scales (Rowe et al 1995). A measure of composite reliability (r c ) was estimated for each of the six scales using WLS regression estimates and error variance estimates from the LISREL output (Rowe 2006;Tarkkonen and Vehkalahti 2005).…”

Methods in School Effectiveness Research∗

“…His view (Chapter 6) is that only 'structured teaching' and 'effective learning time' have received adequate empirical support as factors associated with effectiveness. Similarly, Rowe, Hill, and Holmes-Smith (1995) emphasise that current policy initiatives are poorly supported by the available evidence, and that clear messages are yet to emerge from school effectiveness research. Two of the same authors (Hill & Rowe, 1996) also point out that inadequate attention has generally been paid to the choice and quality of outcome measures.…”

“…This implies very long term data collection strategies and these in turn will require the modelling of more complex cross classified structures because of factors such as mobility (see below). Rowe et al (1995), as well as other authors, point out that since learning takes place within classrooms as well as within schools, the classroom level is a key one to model. If this is done it will usually imply that measurements are taken at least at the start and end of each school year when students enter and leave classes.…”

Situated Ethics in Educational Research

“…A number of the methodological weaknesses with school effectiveness research are described by Rowe and Hill (1995). They point to the failure of many studies to account for the inherently hierarchical or multi-level sampling structure of the data.…”