Lessons Learned from PISA: A Systematic Review of Peer-Reviewed Articles on the Programme for International Student Assessment

Abstract

International large-scale assessments are on the rise, with the Programme for International Student Assessment (PISA) seen by many as having strategic prominence in education policy debates. The present article reviews PISA-related English-language peer-reviewed articles from the programme's first cycle in 2000 to its most current in 2015. Five literature bases were searched, and results were analysed with SPSS. Results map the frequency of publications according to journal, country, and scientific discipline. They also summarise major themes within three identified categories: secondary analysis, policy impact, and critiques. Findings indicated that studies based on the PISA dataset has led to progress in educational research while simultaneously pointing to the need for caution when using this research to inform educational policy.

An Examination of the Influence of International Large Scale Assessments and Global Learning Metrics on National School Reform Policies

“…That is, while there were publications documenting the outcomes of these ILSAs, the literature we reviewed did not specifically address how these other major ILSAs had made an impact on national educational policy or reforms efforts. This finding may be associated with the global prominence of PISA and OECD's recommendations to countries aspiring to improve their PISA results, as well as the much publicized "PISA Shock" (e.g., Hopfenbeck et al, 2017;OECD, 2014;Waldow, 2009;Young, 2015) some countries have experienced.…”

“…Based on the media attention ILSAs receive, particularly after the release of PISA results and international rankings (Baird et al, 2016;Meyer & Benavot, 2013), it appears as though the increased participation in ILSA-related projects and studies has had a profound effect on education and policy discourses globally (Breakspear, 2012;Hopfenbeck et al, 2017), but not necessarily in the ways that make direct causal policy linkages easy to identify. 12 Our review of the literature indicates that some governments appear to have implemented policy changes as a result of ILSAs scores, while others seem to have used these scores to justify and/or accelerate pre-existing school reforms.…”

The politics of PISA: The media, policy and public responses in Norway and England

“…The review concluded that the PISA data‐sets had led to progress in educational research in certain areas, but the authors equally pointed to the need for caution when using this research to inform educational policy. A total of 144 articles was found in the category ‘policy impact’, which focused mainly on the effect of PISA on policy and governments in different countries and economies, with an analysis of the different mechanisms and networks that drove the influence on policy (Hopfenbeck et al, , p.13). Some articles discussed the effect on several countries (Grek, ; Sellar & Lingard, ), whilst others focused on individual country cases such as Germany (Klemm, ; Ertl, ; Di Fuccia, Witteck, Markic & Eilks, ).…”

“…The results are presented and discussed in relation to previous research in both countries and indicate questions for future research. In a recent review of 654 peer-reviewed articles on PISA, three broad categories were documented for the use of PISA data in educational research: 1) secondary analysis of PISA data, 2) policy impact, and 3) critique (Hopfenbeck, Lenkeit, El Masri, Cantrell, Ryan & Baird, 2017). The review concluded that the PISA data-sets had led to progress in educational research in certain areas, but the authors equally pointed to the need for caution when using this research to inform educational policy.…”

High-Level Comprehension Skill by using Competencies PISA in Indonesia's Education System

“…However, Indonesia still far behind Singapore, the first country in the 2015 PISA assessment score in the world. A country can lead the PISA ranking with research progress in the field of education that relies on the frequency of publications according to the journal, country, and scientific discipline [11].…”