Validating the Slovenian national alignment to CEFR: The case of the B2 reading comprehension examination in English

Abstract

The paper examines the results of the CEFR alignment project for the Slovenian national examinations in English. The authors aim to validate externally the standard-setting procedures by adopting a socio-cognitive model of validation (Khalifa & Weir, 2009; Weir, 2005) to analyse the scoring, context and cognitive validity of three reading subtests: the Slovenian B2 national examination and the international examinations FCE and CAE, aligned with B2 and C1 respectively. The relative comparability between the three subtests is determined by analysing the results of tests that have been administered to a group of 80 test-takers (expected CEFR level: B2). The placement of the test-takers also reveals to what extent the judgements of the Slovenian panellists about CEFR levels coincide with those reported for FCE and CAE. The study thus also explores whether the high degree of agreement between the judges on the alignment panel can be solely attributed to their adequate and precise understanding of CEFR descriptors – which is directly mirrored in their setting of the cut scores and relating the examination to relevant CEFR levels – or whether it can also be ascribed to their shared educational, national and cultural background. The answers to these questions are paramount because they reveal the descriptive adequacy of CEFR descriptors and because different interpretations of CEFR levels can significantly affect national testing policies and, consequently, language teaching and testing.

A qualitative inquiry into the application of 9th grade EFL program in terms of the CEFR

“…There have also been studies which have addressed specifically alignment processes of the CEFR into national language education contexts. Ilc and Stopar (2015), for example, addressed the alignment process of national examinations conducted in English to the CEFR proficiency descriptors in Slovenian context. The study reported significance of this kind of processes since understanding descriptors in international benchmarks such as the CEFR may contribute substantially to language teaching policies followed by countries.…”

Relating the Slovenian secondary school English language national examinations to the CEFR: findings and implications

“…Pižorn (2009) also reports on the initial problems the panellists faced at the early stages of the alignment process. However, as partial external evaluations of the alignment project (Ilc and Stopar 2014) suggest, these factors had an insignificant effect on their judgements.…”

Reading for a Test: The Effect of High-Stakes Exams on Reading Strategies

“…Additionally, the same tasks have been aligned with the CEFR level B2, with the cut score of 75 percent. The alignment has also been externally validated (Ilc and Stopar, 2015), and the test has also been studied with regard to item and task difficulty (Stopar and Ilc, 2016). The cut score for B1 was not set during the alignment project; therefore, we label the participants below the B2 cut score as 'sub-B2' readers.…”