RevUP: Automatic Gap-Fill Question Generation from Educational Texts

Kumar, Girish; Banchs, Rafael E.; D’Haro, Luis Fernando

doi:10.3115/v1/w15-0618

Cited by 49 publications

(48 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Models with recent advances of deep learning techniques are even capable of exceeding human performance in some RC tasks, such as for questions with span-based answers ). However, it is not the case when directly applying the state-of-the-art models to multiple choice questions (MCQs) in RACE dataset (Lai et al 2017), elaborately designed by human experts for real examinations, where the task is to select the correct answer from a few given options after reading the article. The performance gap between the state-of-the-art deep models (53.3%) (Tay, Tuan, and Hui 2018) and ceiling (95%) (Lai et al 2017) is significant.…”

Section: Introductionmentioning

confidence: 99%

“…First, a distractor candidate set is extracted from multiple sources, such as GloVe vocabulary (Pennington, Socher, and Manning 2014), noun phrases from textbooks (Welbl, Liu, and Gardner 2017) and articles (Araki et al 2016). Then similarity based (Guo et al 2016;Stasaski and Hearst 2017;Kumar, Banchs, and D'Haro 2015;Mitkov 2003;Zhou, Lyu, and King 2012;Yang et al 2011) or learning based (Liang et al 2018Sakaguchi, Arase, and Komachi 2013;Liang et al 2017;Yang et al 2010) algorithms are employed to select the distractors. Another manner is to apply some pre-defined rules to prepare distractors by changing the surface form of some words or phrases (Chen, Liou, and Chang 2006).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generating Distractors for Reading Comprehension Questions from Real Examinations

Gao

King

et al. 2019

AAAI

View full text Add to dashboard Cite

We investigate the task of distractor generation for multiple choice reading comprehension questions from examinations. In contrast to all previous works, we do not aim at preparing words or short phrases distractors, instead, we endeavor to generate longer and semantic-rich distractors which are closer to distractors in real reading comprehension from examinations. Taking a reading comprehension article, a pair of question and its correct option as input, our goal is to generate several distractors which are somehow related to the answer, consistent with the semantic context of the question and have some trace in the article. We propose a hierarchical encoderdecoder framework with static and dynamic attention mechanisms to tackle this task. Specifically, the dynamic attention can combine sentence-level and word-level attention varying at each recurrent time step to generate a more readable sequence. The static attention is to modulate the dynamic attention not to focus on question irrelevant sentences or sentences which contribute to the correct option. Our proposed framework outperforms several strong baselines on the first prepared distractor generation dataset of real reading comprehension questions. For human evaluation, compared with those distractors generated by baselines, our generated distractors are more functional to confuse the annotators.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generating Distractors for Reading Comprehension Questions from Real Examinations

Gao

King

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…Several earlier works process documents as individual sentences using syntactic (Heilman and Smith, 2010;Ali et al, 2010;Kumar et al, 2015) or semantic-based parsing (Mannem et al, 2010;Lindberg et al, 2013), then reformulate questions using hand-crafted rules acting on parse trees. These traditional approaches generate questions with a high word overlap with the original text that pertain specifically to the given sentence by re-arranging the sentence parse tree.…”

Section: Related Workmentioning

confidence: 99%

Machine Comprehension by Text-to-Text Neural Question Generation

Yuan

Wang

Gülçehre

et al. 2017

Proceedings of the 2nd Workshop on Representation Learning for NLP

145

View full text Add to dashboard Cite

We propose a recurrent neural model that generates natural-language questions from documents, conditioned on answers. We show how to train the model using a combination of supervised and reinforcement learning. After teacher forcing for standard maximum likelihood training, we fine-tune the model using policy gradient techniques to maximize several rewards that measure question quality. Most notably, one of these rewards is the performance of a questionanswering system. We motivate question generation as a means to improve the performance of question answering systems. Our model is trained and evaluated on the recent question-answering dataset SQuAD.

show abstract

“…A good distractor could be a synonym of the key phrase or an important term in the domain of the key phrase. Distractors in Revup, an AQG, are selected from word2vec, a vector of words 16 . Text summarization features like length of a sentence, number of common tokens, number of noun and pronouns, and position of a sentence are generally considered 17 .…”

Section: Gap-fill Questionsmentioning

confidence: 99%

Automatic Question Generation Approaches and Evaluation Techniques

Divate¹,

Salgaonkar²

2017

Current Science

View full text Add to dashboard Cite

The objective of this article is to review several automatic question generation systems and find why automatic question generation is still an attraction for researchers. The focus is mainly on the task of question generation, analysis of the approaches and evaluation of various methods of automatic question generation. Pointers for further research are included.Keywords: Automatic question generation, evaluation techniques, quality enhancers, ranking, sentence simplification.A study reveals that an average student asks over 26 questions per hour in one-on-one human tutoring sessions; in contrast, the student poses 120 questions per hour in a learning environment that forces her to ask questions in order to access any information 1 . Conversely, students learn more deeply if prompted by questions 2 . Conventionally, questions are constructed and assessed by tutors. It has been a trend for several decades that automatic question generation (AQG) system generates questions from the corpora using natural language processing.AQG systems were first developed in the 1976 (ref.3). They have been created for English language and vocabulary, medicine, education and using multimedia. The sequence of developments is as follows: learning words [4][5][6] , English 7 , grammar testing 8 , medicine 9 , academic writing 10 , literature review 11 , education 12 (henceforth, Heilman and Smith AQG is abbreviated as HSAQG), multimedia 13 , and finally a recent major development, on-line learning 14 . This article presents a review of more than 50 contributions in the domain of AQG. Types of questionsIn classroom practice, a tutor evaluates the comprehension of a learner by asking gap-fill type questions (GFQs), multiple choice questions (MCQs), factoid-based questions (FBQs) and deep learning-type questions (DLQs). Gap-fill questionsA stem is a good question or problem to be solved 15 . To identify a stem and generate a GFQ, an informative sentence is selected from a given document. The selection of information involves identification of semantic features in the entire document.Next, a key phrase or answer phrase (assume it is a noun phrase) is selected; term frequency plays an important role. A distractor (not expected to occur in the question) is a choice given to a learner. A good distractor could be a synonym of the key phrase or an important term in the domain of the key phrase. Distractors in Revup, an AQG, are selected from word2vec, a vector of words 16 . Text summarization features like length of a sentence, number of common tokens, number of noun and pronouns, and position of a sentence are generally considered 17 .

show abstract

RevUP: Automatic Gap-Fill Question Generation from Educational Texts

Cited by 49 publications

References 7 publications

Generating Distractors for Reading Comprehension Questions from Real Examinations

Generating Distractors for Reading Comprehension Questions from Real Examinations

Machine Comprehension by Text-to-Text Neural Question Generation

Automatic Question Generation Approaches and Evaluation Techniques

Contact Info

Product

Resources

About