Distractor Generation for Multiple Choice Questions Using Learning to Rank

Liang, Chen; Yang, Xiao; Dave, Neisarg; Wham, Drew C.; Pursel, Bart; Giles, C. Lee

doi:10.18653/v1/w18-0533

Cited by 55 publications

(48 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Faizan and Lohmann (2018) and Faizan et al (2017) and Stasaski and Hearst (2017) adopted a similar approach for selecting distractors. Others, including Liang et al (2017Liang et al ( , 2018 and Liu et al (2018), used ML-models to rank distractors based on a combination of the previous features. Again, some distractor selection approaches are tailored to specific types of questions.…”

Section: Generation Tasksmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Automatic Question Generation for Educational Purposes

Kurdi

Leo

Parsia

et al. 2019

Int J Artif Intell Educ

267

111

View full text Add to dashboard Cite

While exam-style questions are a fundamental educational tool serving a variety of purposes, manual construction of questions is a complex process that requires training, experience, and resources. This, in turn, hinders and slows down the use of educational activities (e.g. providing practice questions) and new advances (e.g. adaptive testing) that require a large pool of questions. To reduce the expenses associated with manual construction of questions and to satisfy the need for a continuous supply of new questions, automatic question generation (AQG) techniques were introduced. This review extends a previous review on AQG literature that has been published up to late 2014. It includes 93 papers that were between 2015 and early 2019 and tackle the automatic generation of questions for educational purposes. The aims of this review are to: provide an overview of the AQG community and its activities, summarise the current trends and advances in AQG, highlight the changes that the area has undergone in the recent years, and suggest areas for improvement and future opportunities for AQG. Similar to what was found previously, there is little focus in the current literature on generating questions of controlled difficulty, enriching question forms and structures, automating template construction, improving presentation, and generating feedback. Our findings also suggest the need to further improve experimental reporting, harmonise evaluation metrics, and investigate other evaluation methods that are more feasible.

show abstract

Section: Generation Tasksmentioning

confidence: 99%

“…The science exam questions were identified in 55% of the cases. This corpus was used by Liang et al (2018) to develop and test a model for ranking distractors. All keys and distractors in the dataset were fed to the model to rank.…”

Section: Standard Datasetsmentioning

confidence: 99%

A Systematic Review of Automatic Question Generation for Educational Purposes

Kurdi

Leo

Parsia

et al. 2019

Int J Artif Intell Educ

267

111

View full text Add to dashboard Cite

show abstract

“…(Flor & Riordan, 2018) Question NLP Generation This paper presents a novel rule-based system for automatic generation of factual questions from sentences, using semantic role labeling (SRL) as the main form of text analysis. (Liang et al, 2018) Question…”

Section: Othersmentioning

confidence: 99%

Text mining in education

Mello

Andre

Pinheiro

et al. 2019

WIREs Data Min & Knowl

148

View full text Add to dashboard Cite

The explosive growth of online education environments is generating a massive volume of data, specially in text format from forums, chats, social networks, assessments, essays, among others. It produces exciting challenges on how to mine text data in order to find useful knowledge for educational stakeholders. Despite the increasing number of educational applications of text mining published recently, we have not found any paper surveying them. In this line, this work presents a systematic overview of the current status of the Educational Text Mining field. Our final goal is to answer three main research questions: Which are the text mining techniques most used in educational environments? Which are the most used educational resources? And which are the main applications or educational goals? Finally, we outline the conclusions and the more interesting future trends. This article is categorized under: Application Areas > Education and Learning Ensemble Methods > Text Mining

show abstract

“…A method to generate distractors for fill-in-the-blank questions was proposed in [23]. In [13], the authors use feature-based ensemble and neural net-based models to rank distractors.…”

Section: Related Workmentioning

confidence: 99%

Learning to Distract: A Hierarchical Multi-Decoder Network for Automated Generation of Long Distractors for Multiple-Choice Questions for Reading Comprehension

Maurya

Desarkar

2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

The task of generating incorrect options for multiple-choice questions is termed as distractor generation problem. The task requires high cognitive skills and is extremely challenging to automate. Existing neural approaches for the task leverage encoder-decoder architecture to generate long distractors. However, in this process two critical points are ignored-firstly, many methods use Jaccard similarity over a pool of candidate distractors to sample the distractors. This often makes the generated distractors too obvious or not relevant to the question context. Secondly, some approaches did not consider the answer in the model, which caused the generated distractors to be either answer-revealing or semantically equivalent to the answer. In this paper, we propose a novel Hierarchical Multi-Decoder Network (HMD-Net) consisting of one encoder and three decoders, where each decoder generates a single distractor. To overcome the first problem mentioned above, we include multiple decoders with a dis-similarity loss in the loss function. To address the second problem, we exploit richer interaction between the article, question, and answer with a SoftSel operation and a Gated Mechanism. This enables the generation of distractors that are in context with questions but semantically not equivalent to the answers. The proposed model outperformed all the previous approaches significantly in both automatic and manual evaluations. In addition, we also consider linguistic features and BERT contextual embedding with our base model which further push the model performance. CCS CONCEPTS • Computing methodologies → Natural language generation.

show abstract

Distractor Generation for Multiple Choice Questions Using Learning to Rank

Cited by 55 publications

References 28 publications

A Systematic Review of Automatic Question Generation for Educational Purposes

A Systematic Review of Automatic Question Generation for Educational Purposes

Text mining in education

Learning to Distract: A Hierarchical Multi-Decoder Network for Automated Generation of Long Distractors for Multiple-Choice Questions for Reading Comprehension

Contact Info

Product

Resources

About