Use of word lists in a high‐stakes, low‐exposure context: Topic‐driven or frequency‐informed

Abstract

The awarding organizations that create and administer high‐stakes assessments for beginner‐to‐low‐intermediate 16‐year‐old learners of French, German, and Spanish in England provide optional topic‐driven word lists as guides for teachers and textbook writers. Given that these lists are developed by the awarding organizations, they exert a powerful washback effect on teaching and learning. However, we do not know how much of these lists have actually been used in exams. We therefore analyzed the extent to which these lists have been used when developing the General Certificate of Secondary Education listening and reading exams, a corpus totaling 116,647 words. One key finding showed that approximately half of the awarding organizations’ lists had never been used in any of the exams to date. Given recent changes to curriculum policy, we also investigated how word list type—frequency‐informed versus the awarding organizations’ topic‐driven lists—affected lexical coverage of the exams. Overall, our findings suggested that using the topic‐driven lists was likely to be a suboptimal use of lesson time, as they did not provide learners with enough words to understand any given text with ease. Frequency‐informed word lists, however, seemed to better prepare learners for the exams.

The lexical content of high‐stakes national exams in French, German, and Spanish in England

“…Clearly, other factors, such as "semantic neutrality, length, part of speech, polysemy, morphological regularity, cognateness, [and] orthographic transparency" (Hashimoto, 2021, p. 182), also influence lexical difficulty, and the difficulty of any written or spoken text involves many dimensions beyond the lexicon. Similarly, objective word frequency information (e.g., corpus-based data and lexical coverage) should not be the sole criterion for wordlist development, which can helpfully also draw on subjective criteria (e.g., teacher evaluations; Dang et al, 2020;He & Godfroid, 2019;Marsden et al, 2023).…”

The Lexical Content of High-Stakes National Exams in French, German, and Spanish in England

“…Clearly, other factors, such as "semantic neutrality, length, part of speech, polysemy, morphological regularity, cognateness, [and] orthographic transparency" (Hashimoto, 2021, p. 182), also influence lexical difficulty, and the difficulty of any written or spoken text involves many dimensions beyond the lexicon. Similarly, objective word frequency information (e.g., corpus-based data and lexical coverage) should not be the sole criterion for wordlist development, which can helpfully also draw on subjective criteria (e.g., teacher evaluations; Dang et al, 2020;He & Godfroid, 2019;Marsden et al, 2023). Assuming that it is not possible to substantially increase curriculum time given to French, German, or Spanish, then more clearly defining the lexical content for GCSE exams, with a higher concentration of high-frequency words, could help tighten the link between 'what is taught' and 'what can be assessed' and allow students to get more return for their learning effort (Laufer & Nation, 2012;Nation, 2013).…”

A Context-Aligned Two Thousand Test: Towards estimating high-frequency French vocabulary knowledge for beginner-to-low intermediate proficiency adolescent learners in England

“…Moreover, much of this vocabulary is likely to be mid-to-low frequency: Marsden, Dudley, and Hawkes (2023), for instance, reported that of the 1,322 flemmas on AQA's (the leading awarding organisation in England) current GCSE French wordlist, only 48% were highfrequency. Thus, any test of vocabulary knowledge that randomly samples 20 or even 30 words from each band is unlikely to provide a valid or useful measure of these students' vocabulary knowledge.…”