Effects of International Comparative Studies on Educational Quality on the Quality of Educational Research

Abstract

Large-scale survey studies of educational achievement are becoming increasingly frequent, and they are visibly present in both educational policy debates and within the educational research community. One main aim of these studies is to provide descriptions of inputs, processes and outcomes, and another aim is to provide explanations of how different factors interrelate to produce educational outcomes. These aims are difficult to reach, which in combination with the fact that the comparative studies are typically more policy driven than theory driven, are reasons why these studies are contested on quality grounds. In this article, a set of fundamental methodological challenges related to the validity of the measurement instruments and to the possibility of making inferences about causality are identified and discussed in relation to examples of different studies. Strengths and weaknesses of different research approaches are discussed, and it is proposed that the dichotomy between qualitative and quantitative approaches should be replaced with distinctions between lowand high-level inference approaches with respect to data, generalization and explanation. It is concluded that while the international studies easily invite misuse and misinterpretation, they also offer possibilities for improving the quality of educational research, because the high-quality data generated by these studies can be taken advantage of in research on causal effects of factors in and out of educational systems.

Is computer availability at home causally related to reading achievement in grade 4? A longitudinal difference in differences approach to IEA data from 1991 to 2006

“…However, aggregation of data brings other advantages as well. For example, mechanisms that, at the individual level, are responsible for reverse causality, such as compensatory resource allocation, are unlikely to be present at the country level (Gustafsson 2008). Aggregated data also have the advantage of not being as severely influenced by errors of measurement, compared to individual data.…”

Uticaj socio-ekonomskog statusa učenika na obrazovna postignuća: direktni i indirektni uticaji

“…Međunarodni program procene učeničkih postignuća PISA (Programme for International Student Assessment) je u ovom trenutku jedno od najrelevantnijih međunarodnih istraživanja u domenu obrazovnih nauka (Baucal, 2012;Baumert, Blum, & Neubrand, 2002;Gustafsson, 2008;OECD, 2010). Osnovni cilj PISA studije je da se sistematski prati u kojoj meri su učenici na uzrastu 15 godina razvili tri ključne kompetencije (čitalačka, naučna i matematička pismenost).…”

Instructional responsibility in mathematics education: modelling classroom teaching using Swedish data

“…This thus constitutes a restriction when interpreting the validity of the construct. However, the TIMSS data offer a lot of information and the quality is excellent in many respects (Gustafsson, 2008), although the complexity, the lack of control of the data and the loss of key variables should be taken into account when performing secondary analysis (Bryman, 2004).…”