League tables in educational evidence-based policy-making: can we stop the horse race, please?

Klemenčič, Eva; Mirazchiyski, Plamen

doi:10.1080/03050068.2017.1383082

Cited by 20 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondary analyses of the data can reveal how different aspects of the curricula, organization of instruction, background characteristics, and the general context of education can impact the educational outcomes. Analysis of these data serves as a tool for making new or changing existing policies, starting and/or supporting ongoing reforms (Klemenčič and Mirazchiyski, 2018;Wiseman, 2010). Given the importance of the decisions based on the results, ILSAs require special tools for analyzing their data due to the methodological and statistical complexities accompanying them.…”

Section: Summary and Discussionmentioning

confidence: 99%

RALSA: the R analyzer for large-scale assessments

Mirazchiyski

2021

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

This paper presents the R Analyzer for Large-Scale Assessments (), a newly developed package for analyzing data from studies using complex sampling and assessment designs. Such studies are, for example, the IEA’s Trends in International Mathematics and Science Study and the OECD’s Programme for International Student Assessment. The package covers all cycles from a broad range of studies. The paper presents the architecture of the package, the overall workflow and illustrates some basic analyses using it. The package is open-source and free of charge. Other software packages for analyzing large-scale assessment data exist, some of them are proprietary, others are open-source. However, is the first comprehensive package, designed for the user experience and has some distinctive features. One innovation is that the package can convert SPSS data from large scale assessments into native data sets. It can also do so for PISA data from cycles prior to 2015, where the data is provided in tab-delimited text files along with SPSS control syntax files. Another feature is the availability of a graphical user interface, which is also written in and operates in any operating system where a full copy of can be installed. The output from any analysis function is written into an MS Excel workbook with multiple sheets for the estimates, model statistics, analysis information and the calling syntax itself for reproducing the analysis in future. The flexible design of allows for the quick addition of new studies, analysis types and features to the existing ones.

show abstract

Section: Summary and Discussionmentioning

confidence: 99%

RALSA: the R analyzer for large-scale assessments

Mirazchiyski

2021

Large-scale Assess Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…Examples of such countries are Germany, Poland, Portugal, and Vietnam. After the 2012 PISA results were published (Klemenčič and Mirazchiyski 2017) the British media and government highlighted German, Polish, and Vietnamese score improvements and policies. Portugal also became a projection screen for other countries' ideas after the 2012 survey (Rutkowski 2015), and increasingly after the good results presented in 2015 survey.…”

Section: Pisa and The Construction Of Reference Societiesmentioning

confidence: 99%

Inspirations from abroad: the impact of PISA on countries’ choice of reference societies in education

Santos

Centeno

2021

Compare: A Journal of Comparative and International Education

View full text Add to dashboard Cite

The Programme of International Student Assessment (PISA) has instigated domestic policy debate aimed at improving education systems' quality and efficiency. Its high performers are often described as knowledge-based legitimation tools that have become reference societies. This article analyses if and the extent to which PISA affects the choice of reference societies used in education policy. We conducted a systematic literature review of scholarly peer-reviewed journal articles. We identify two ways in which PISA affects the reference societies chosen: it builds but also triggers the collapse of the reference societies used in domestic education policy. However, there are also cases in which PISA has little influence on which countries or regions are used as references. The domestic processes of choosing the reference societies used in education policy emerge as more nuanced than first expected. This article provides a firm basis for a much-needed understanding of the topic.

show abstract

“…Without a doubt, the country rankings based on ILSA results have gained considerable attention in many countries. Klemenčič and Mirazchiyski (2018) argue that league tables should not be perceived as the ultimate product of ILSAs and the responsibility for interpretation lies with the researchers' working on the ILSA data. While the results of ILSAs may be used as snapshots for countries' performances, the longitudinal component on the country level warrant analyses with a larger scope.…”

Section: Introductionmentioning

confidence: 99%

Measures of long-term trends in mathematics: linking large-scale assessments over 50 years

et al. 2021

View full text Add to dashboard Cite

International comparative assessments of student achievement are constructed to assess country-level differences and change over time. A coherent understanding of the international trends in educational outcomes is strongly needed as suggested by numerous previous studies. Investigating these trends requires long-term analysis, as substantial changes on the system level are rarely observed regarding student outcomes in short periods (i.e., between adjacent international assessment cycles). The present study aims to link recent and older studies conducted by the International Association for the Evaluation of Educational Achievement (IEA) onto a common scale to study long-term trends within and across countries. It explores the comparability of the achievement tests of the Trends in International Mathematics and Science Study and previous IEA studies on mathematics in grade eight. Employing item response theory, we perform a concurrent calibration of item parameters to link the eight studies onto a common scale spanning the period from 1964 to 2015 using data from England, Israel, Japan, and the USA.

show abstract

League tables in educational evidence-based policy-making: can we stop the horse race, please?

Cited by 20 publications

References 18 publications

RALSA: the R analyzer for large-scale assessments

RALSA: the R analyzer for large-scale assessments

Inspirations from abroad: the impact of PISA on countries’ choice of reference societies in education

Measures of long-term trends in mathematics: linking large-scale assessments over 50 years

Contact Info

Product

Resources

About