Can test construction account for varying gender differences in international reading achievement tests of children, adolescents and young adults? – A study based on Nordic results in PIRLS, PISA and PIAAC

Abstract

Gender differences in reading seem to increase throughout schooling and then decrease or even disappear with age, but the reasons for this are unclear. In this study, we explore whether differences in the way 'reading literacy' is operationalised can add to our understanding of varying gender differences in international large-scale surveys. We first compare the impact of gender on reading literacy in PIRLS 2011 (10-year-olds), Programme for International Student Assessment (PISA) 2009 (15-year-olds) and Programme for the International Assessment of Adult Competencies (PIAAC) 2012 (16-24-year-olds), respectively, across the Nordic countries. Then we compare how reading literacy is operationalised in those surveys. We find similar patterns of gender differences across the Nordic countries, with the largest effect sizes in PISA and the smallest in PIAAC. Further, even though the three surveys define reading literacy in similar ways, they operationalise and assess that construct differently. The magnitude of the observed gender differences appears to be associated with certain assessment features including text type, item format, aspects of reading and implementation.

Doing research with international assessment studies: methodological and conceptual challenges and ways forward

“…They ask if this means that gender differences decrease with advancing age or if surveys measure different skills. Solheim and Lundetrae (2018) find that while PIRLS, PISA and PIAAC conceptualise reading literacy in similar ways, their operationalisation of the construct and test design is very different indeed. Gender differences across the different age groups and surveys appear -at least to some extent -to be associated with features of the assessment that include text-types (fiction and non-fiction), item format (multiple-choice or constructed item) and aspects of the implementation (classrooms or home administration).…”

“…Articles by Shepherd (2018) and Solheim and Lundetrae (2018) focus indeed on language and reading, although not necessarily underlying cognitive processes. Shepherd (2018) examines the effect of exposure to mother-tongue and English instruction on student reading performances in Botswana and South Africa.…”

“…The article represents an insightful evaluation of language policy effects on two linguistically and culturally similar groups of students on each side of the border between South Africa and Botswana. Solheim and Lundetrae (2018) raise the question whether test construction in different international studies can account for varying gender differences in reading attainment for children, adolescents and young adults in the Nordic countries. They show that reading differences seem to be largest for adolescents taking the PISA test but are smallest for young adults in PIAAC.…”

10 Gutte- og jentelesere i topp og bunn. En utforsking av leserprofiler

“…Jenter rapporterer også å vaere mer motivert for lesing enn gutter (Brozo et al, 2014;OECD, 2010b). Kjønnsforskjellene ser ut til å øke gjennom grunnskolen og er størst i PISA (Solheim & Lundetrae, 2016). Likevel finner vi både gutter og jenter blant de beste leserne så vel som blant de svakeste leserne på tvers av alderstrinn.…”

“…Selv om elevene på samme mestringsnivå har relativt lik total leseskår i PIRLS 2016, kan variasjonen vaere større innad på mestringsnivået når det gjelder hvordan de skårer på ulike type tekster og ulike typer spørsmål. Tidligere studier har funnet at kjønnsforskjellen i jentenes favør er størst når elevene leser skjønnlitteraere tekster (Solheim & Lundetrae, 2016). I tillegg gjør jentene det ofte bedre enn guttene på oppgaver der de må bruke egne erfaringer og reflektere over det de har lest, spesielt når de må formulere og skrive svaret selv (Lafontaine & Monseur, 2009;Roe & Taube, 2003;Schwabe, McElvany & Trendtel, 2015).…”