The impact of test language on PISA scores. New evidence from Wales

Abstract

In this paper we investigate the link between the language in which pupils take the Programme for International Student Assessment (PISA) test and the scores they achieve in this assessment, focusing upon the case of Wales. Using five rounds of PISA data and an instrumental variable approach, we show how pupils who took the test in Welsh score around 0.3 standard deviations (30 PISA test points) lower in reading, mathematics and science than their peers who took the test in English. This finding is robust to different model specifications and statistical approaches. We argue that this may indicate that the academic performance of teenagers in Wales may be underestimated in PISA -particularly amongst those who take this test in Welsh. K E Y W O R D Shome language, PISA, test language, Wales | 421 THE IMPACT OF TEST LANGUAGE ON PISA SCORES. NEW EVIDENCE FROM WALES Notes: All OECD recommended practices (final student weights and balanced repeated replication (BRR) weights) have been employed. The "-" indicates that there are no data for that region in the PISA cycle.

Do more effective schools succeed in reducing the gender gap in linguistic communicative competence in a bilingual context? (¿Consiguen los centros más eficaces reducir la brecha de género en competencia comunicativa lingüística en un contexto bilingüe?)

“…Research into student performance in bilingual contexts has mainly focused on analysing possible differences in accordance with mother tongue, based on the combination of family language and instruction language or language competence assessment instruments, in both the ACBC (Doncel Abad & Cabrera-Álvarez, 2020; Elosua & De Boek, 2020) and other bilingual contexts (Jerrim et al, 2021; Shorbagi et al, 2022). None, however, has focused principally on the gender gap.…”

“…La investigación sobre el rendimiento del alumnado en contextos bilingües se ha centrado más en conocer las posibles diferencias en base a la lengua materna, basada en la combinación de la lengua familiar y la lengua de instrucción o los instrumentos de evaluación de la competencia lingüística, tanto en lengua vasca (Doncel Abad & Cabrera-Álvarez, 2020; Elosua & De Boek, 2020) u otros contextos bilingües (Jerrim et al, 2021; Shorbagi et al, 2022) pero no se han centrado en considerar la brecha de género. Otras investigaciones han estudiado las actitudes hacia los idiomas minoritarios indicando que las chicas manifiestan actitudes más positivas en el caso del galés (Price & Tamburelli, 2020), gallego (Loureiro-Rodriguez et al, 2013), irlandés (McCoy et al, 2012) o catalán (Ubalde et al, 2017).…”

“…Es necesario, por tanto, explorar la brecha de género en la competencia en comunicación lingüística en mayor detalle, teniendo en cuenta el contexto sociocultural y lingüístico, ya que, tanto en estudios nacionales (Artola et al, 2017 2015) como internacionales (Hadjar et al, 2014;Reilly et al, 2019;Scheiber et al, 2015) se ha encontrado que las chicas obtienen puntuaciones más altas en la competencia en comunicación lingüística que los chicos, pero estos resultados puede que no se puedan aplicar a contextos bilingües. La investigación sobre el rendimiento del alumnado en contextos bilingües se ha centrado más en conocer las posibles diferencias en base a la lengua materna, basada en la combinación de la lengua familiar y la lengua de instrucción o los instrumentos de evaluación de la competencia lingüística, tanto en lengua vasca (Doncel Abad & Cabrera-Álvarez, 2020;Elosua & De Boek, 2020) u otros contextos bilingües (Jerrim et al, 2021;Shorbagi et al, 2022) pero no se han centrado en considerar la brecha de género. Otras investigaciones han estudiado las actitudes hacia los idiomas minoritarios indicando que las chicas manifiestan actitudes más positivas en el caso del galés (Price & Tamburelli, 2020), gallego (Loureiro-Rodriguez et al, 2013), irlandés (McCoy et al, 2012 o catalán (Ubalde et al, 2017).…”

Major challenges for education in Wales

“…Jerrim ( 2023) has written about the curious issue of the implausibly low scores of pupils taking the test in Welsh, which could be biasing Welsh scores downwards. However, such biases are likely to be modest (less than 10 points) and have been known to affect previous years (Jerrim, Lopez-Agudo and Marcenaro-Gutierrez, 2022).…”

Problem Types Used by Content Areas: A Comparative Analysis on PISA and National Exams

“…Dolayısıyla PISA, dünyada ve ülkemiz eğitim otoriteleri tarafından da önemsenmekte, eğitim sistemimiz dinamikleri sık sık PISA ile karşılaştırılmaktadır. Örneğin güncel alanyazın incelendiğinde; ülkeler arası akademik başarı bağlamında karşılaştırmaların yapıldığı (Boman, 2022;Üstün, Cansız, Özdemir ve Cansız, 2022;Marquez, Lambert, Ridge ve Walker, 2022;Erdoğdu, 2022), farklı uygulamaların PISA performansına etkilerinin araştırıldığı (Navarro-Martinez ve Peña-Acuña, 2022; Jerrim, Lopez-Agudo ve Marcenaro-Gutierrez, 2022;Dong ve Kula, 2022;Liu ve Wang, 2022), öğrencilere ve eğitim sistemlerine ilişkin bazı niteliklerin PISA performanslarına etkisinin araştırıldığı (Qian ve Lau, 2022;Khine, Fraser, Afari ve Liu, 2022;Yang, Fan ve Chen, 2022;İnci ve Kaya, 2022;Kaya, Eryilmaz ve Yüksel, 2022;Finch ve Finch, 2022) çalışmalara sıklıkla rastlanmaktadır. Ancak yapılan karşılaştırmalar, görüldüğü üzere çoğunlukla test sonuçları üzerinden olmakta ve akademik başarıya odaklanmaktadır.…”