Parametrizable exercise generation from authentic texts: Effectively targeting the language means on the curriculum

Abstract

We present a parametrizable approach to exercise generation from authentic texts that addresses the need for digital materials designed to practice the language means on the curriculum in a real-life school setting. The tool builds on a language-aware search engine that helps identify attractive texts rich in the language means to be practiced. Making use of state-ofthe-art NLP, the relevant learning targets are identified and transformed into exercise items embedded in the original context.While the language-aware search engine ensures that these contexts match the learner's interests based on the search term used, and the linguistic parametrization of the system then reranks the results to prioritize texts that richly represent the learning targets, for the exercise generation to proceed on this basis, an interactive configuration panel allows users to adjust exercise complexity through a range of parameters specifying both properties of the source sentences and of the exercises.An evaluation of exercises generated from web documents for a representative sample of language means selected from the English curriculum of 7th grade in German secondary school showed that the combination of language-aware search and exercise generation successfully facilitates the process of generating exercises from authentic texts that support practice of the pedagogical targets.

The instructional effectiveness of automatically generated exercises for learning French grammatical gender: preliminary results

“…Finally, the present study focused on a single aspect of learning a foreign language, grammatical gender, over a relatively short time (three practice sessions). To obtain more support for the instructional effectiveness of AEG-based instruction in general, it would be interesting to carry out an evaluation with a system that supports a variety of linguistic targets, such as the system developed by Heck and Meurers (2022), over a longer period of time and with more participants.…”

“…Among the existing methods for exercise generation (see Perez-Beltrachini et al, 2012 for a discussion of different methods), our approach has the most in common with the systems developed by Heift and Toole (2002) and Heck and Meurers (2022) as our pipeline relies on NLP tools to handle arbitrary documents as input (as opposed to being based on static corpora, e.g. Pilán et al, 2017, or automatically generated language, e.g.…”

“…Perez-Beltrachini et al, 2012;Verweij, 2020). Our approach differs from (Heift and Toole, 2002) because the pipeline requires another NLP component, namely a dependency parser, and unlike the work by Heck and Meurers (2022), our pipeline does not accept HTML but is limited to plain text, as preserving the original look and feel of the document was not an important requirement to realise the instructional approach we selected.…”

Generating and authoring high-variability exercises from authentic texts

“…Closed activity types such as Multiple Choice (MC) are especially popular due to their ability to automatically score the exercises based on the very restricted space of possible learner answers (Tafazoli et al, 2019), yet supported exercise formats vary from one system to the other. A number of tools integrate a variety of different formats: MIRTO automatically generates Fillin-the-Blanks (FiB) as well as Mark-the-Words (MtW) exercises (Antoniadis et al, 2004); Arik-Iturri can generate MC, Error Detection, FiB and Word Formation exercises (Aldabe et al, 2006); an extension of the language aware search Engine FLAIR 1 (Heck and Meurers, 2022b) covers a wide range including FiB, MC, MtW, Memory, Jumbled Sentences and Drag and Drop exercises; Sakumon (Hoshino and Nakagawa, 2008) and Cloze-Fox (Jozef and Sevinc, 2010) support cloze exercises in FiB as well as MC format; WERTi (Meurers et al, 2010) and its multilingual extension View (Reynolds et al, 2014) in addition feature MtW exercises, the Language Exercise App Sentence Shuffling activities (Pérez and Cuadros, 2017), and Ferreira and Pereira Jr. (2018)'s Verb Tenses System True/False and Tense transposition exercises. While these systems can generate multiple exercises for a linguistic structure from the same source document, the actual number of exercises is usually quite limited.…”

“…By varying exercise parameters such as the number of distractors, hints in parentheses, or the span of the target construction, variability can be increased. Notable examples making use of such parameterizations constitute MIRTO which provides parameters for the choice of target constructions, parentheses of FiB exercises and support elements such as reference pages (Antoniadis et al, 2004); the assistant system Sakumon which requires users to manually select target items and distractors from automatically generated suggestions (Hoshino and Nakagawa, 2008); the Language Exercise App where target constructions, distractors and parentheses of FiB exercises are parameterizable (Pérez and Cuadros, 2017); and FLAIR's exercise generation functionality which, in addition to providing parameters for target constructions, distractors and parentheses, allows users to influence the specificity of the exercise instructions (Heck and Meurers, 2022b). However, these systems require users to specify each configuration individually so that generating large numbers of parameterized exercises involves considerable configuration effort as well as manual labour to review the generated exercises for correctness.…”