Semantic Similarity Measures for the Generation of Science Tests in Basque

Abstract

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language Processing (NLP) techniques and the use of corpora. More specifically, human experts select scientific articles and identify the target terms (i.e., words). These terms are part of the vocabulary studied in the school curriculum for 13-14-year-olds and form the starting point for our system to generate MCQs. We automatically generate distractors that are similar in meaning to the target term. To this end, the system applies semantic similarity measures making use of a variety of corpus-based and graph-based approaches. The paper presents a qualitative and a quantitative analysis of the generated tests to measure the quality of the proposed methods. The qualitative analysis is based on expert opinion, whereas the quantitative analysis is based on the MCQ test responses from students in secondary school. Nine hundred and fifty one students from 18 schools took part in the experiments. The results show that our system could help experts in the generation of MCQ.

An Intelligent Testing Strategy for Vocabulary Assessment of Chinese Second Language Learners

“…Automatic item generation (AIG) is a promising approach to reduce the cost of test development. AIG methods have been used in generating different types of questions, such as reading comprehen- sion (Rus et al, 2007;Mostow et al, 2017) and vocabulary assessment (Mitkov et al, 2006(Mitkov et al, , 2009Aldabe and Maritxalar, 2014). Due to its high efficiency and controllability, automatic item generation has been used to create solutions and rationales for Computerized Formative Testing (Gierl and Lai, 2018).…”

An Environment for Automatic Test Generation

“…The strategies can be applied to develop a large set of questions. In [18], a vocabulary-learning scenario in a Basque-language is used to support teachers by generation of multiple choice questions. In this environment, multiple-choice questions are generated semi-automatically applying natural language processing techniques.…”

Testuliburuetatik domeinu-modulu eleaniztunak eraikitzen

“…Domeinu-modulua sortzea zaila bada, are zailagoa da domeinu-modulua era automatikoan edo erdiautomatikoan erauztea, dokumentu elektronikoetatik edo corpusetatik abiatuta. Egiteko horren inguruan argitaratu diren lanak, gainera, domeinu-modulu elebakarretara mugatuta daude [8,9,10,11,12,13]. Euskararen kasuan, DOM-Sortze inguruneak ahalbidetzen du Testuliburuetatik domeinu-modulu eleaniztunak eraikitzen domeinu-moduluak sortzea.…”