Ontology-Based Multiple Choice Question Generation

Abstract

With recent advancements in Semantic Web technologies, a new trend in MCQ item generation has emerged through the use of ontologies. Ontologies are knowledge representation structures that formally describe entities in a domain and their relationships, thus enabling automated inference and reasoning. Ontology-based MCQ item generation is still in its infancy, but substantial research efforts are being made in the field. However, the applicability of these models for use in an educational setting has not been thoroughly evaluated. In this paper, we present an experimental evaluation of an ontology-based MCQ item generation system known as OntoQue. The evaluation was conducted using two different domain ontologies. The findings of this study show that ontology-based MCQ generation systems produce satisfactory MCQ items to a certain extent. However, the evaluation also revealed a number of shortcomings with current ontology-based MCQ item generation systems with regard to the educational significance of an automatically constructed MCQ item, the knowledge level it addresses, and its language structure. Furthermore, for the task to be successful in producing high-quality MCQ items for learning assessments, this study suggests a novel, holistic view that incorporates learning content, learning objectives, lexical knowledge, and scenarios into a single cohesive framework.

Ontology-Based Multiple Choice Question Generation

“…For a detailed review of the related literature, the interested reader is referred to [2]. A number of ontology-based question generation approaches have been proposed [3,6,7,12,15,19,20]. For example, Zitko et al [19] proposed templates and algorithms for the automatic generation of objective questions from ontologies.…”

“…MCQs are, however, rather difficult to design [9], regardless of whether we design them by hand or automatically: in addition to a good question 2 and a correct answer, 3 we need to generate suitable distractors, i.e., reasonable yet incorrect alternative answers. To generate a good set of MCQs, i.e., an exam, we need to control the difficulty of MCQs: a good exam has to have questions from a range of difficulties so that it faithfully assesses students' understanding.…”

“…While such mechanisms can be suitable for generating distractors for language testing, it is clearly not always suitable for other domains. Thus, an interest in semanticsbased distractor generation mechanisms has developed, see for example [3,6,7,14,15]. These approaches utilise some notion of concept similarity to select suitable distractors just as we suggest in the current project, however, on the one hand, they do not examine the correlation between their utilised notion of similarity and human judgements of similarity (since they will be answering the generated questions).…”

A novel approach to generate MCQs from domain ontology: Considering DL semantics and open-world assumption

“…Our approach in this paper is an effort in this direction. We observed that most of the existing MCQ generation techniques [7] randomly select instances which do not belong to the class of the correct answer as distractors. The incorrectness of the distractors cannot be ensured by this random selection method, which in turn made it necessary to manually check the correctness of the question items before making use of them.…”

“…Recently, a handful of studies [3][4][5][6][7][8][9] explored the use of structured domain knowledge in the form of description logic based ontologies to automatically generate MCQs. This would enable online assessment systems to utilize existing knowledge bases for the assessment of learner's knowledge and skills.…”