Neural Question Generation with Semantics of Question Type

Dong, Xiaozheng; Yu, Hong; Chen, Xin; Li, Weikang; Zhang, Min; Zhu, Qiaoming

doi:10.1007/978-3-319-99501-4_18

Cited by 8 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For MCQ generation, target answer is usually given with the reference document, which serves as the answer for the generated question (Vachev et al, 2022). With the advent of deep learning, various neural networks have been used for QG: LSTMbased (Dong et al, 2018) and transformer-based (Laban et al, 2022;Hosking and Riedel, 2019).…”

Section: Multiple-choice Question Generationmentioning

confidence: 99%

Evaluating the Knowledge Dependency of Questions

Hyeongdon¹,

Yang²,

Yu³

et al. 2022

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The automatic generation of Multiple Choice Questions (MCQ) has the potential to reduce the time educators spend on student assessment significantly. However, existing evaluation metrics for MCQ generation, such as BLEU, ROUGE, and METEOR, focus on the n-gram based similarity of the generated MCQ to the gold sample in the dataset and disregard their educational value. They fail to evaluate the MCQ's ability to assess the student's knowledge of the corresponding target fact. To tackle this issue, we propose a novel automatic evaluation metric, coined Knowledge Dependent Answerability (KDA), which measures the MCQ's answerability given knowledge of the target fact. Specifically, we first show how to measure KDA based on student responses from a human survey. Then, we propose two automatic evaluation metrics, KDA disc and KDA cont , that approximate KDA by leveraging pre-trained language models to imitate students' problem-solving behavior. Through our human studies, we show that KDA disc and KDA cont have strong correlations with both (1) KDA and (2) usability in an actual classroom setting, labeled by experts. Furthermore, when combined with ngram based similarity metrics, KDA disc and KDA cont are shown to have a strong predictive power for various expert-labeled MCQ quality measures. 1

show abstract

Section: Multiple-choice Question Generationmentioning

confidence: 99%

Evaluating the Knowledge Dependency of Questions

Hyeongdon¹,

Yang²,

Yu³

et al. 2022

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Question generation (QG) is an emerging research topic due to its wide application scenarios such as education , goal-oriented dialogue (Lee et al, 2018), and question answering . The preliminary neural QG models outperform the rule-based methods relying on hand-craft features, and thereafter various models have been proposed to further improve the performance via incorporating question type (Dong et al, 2018), answer position , long passage modeling (Zhao et al, 2018b), question difficulty , and to the point context (Li et al, 2019). Some works try to find the possible answer text spans for facilitating the learning .…”

Section: Related Workmentioning

confidence: 99%

Review-based Question Generation with Adaptive Instance Transfer and Augmentation

Yu¹,

Zhang²,

Lam³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

While online reviews of products and services become an important information source, it remains inefficient for potential consumers to exploit verbose reviews for fulfilling their information need. We propose to explore question generation as a new way of review information exploitation, namely generating questions that can be answered by the corresponding review sentences. One major challenge of this generation task is the lack of training data, i.e. explicit mapping relation between the user-posed questions and review sentences. To obtain proper training instances for the generation model, we propose an iterative learning framework with adaptive instance transfer and augmentation. To generate to the point questions about the major aspects in reviews, related features extracted in an unsupervised manner are incorporated without the burden of aspect annotation. Experiments on data from various categories of a popular E-commerce site demonstrate the effectiveness of the framework, as well as the potentials of the proposed review-based question generation task.

show abstract

“…Beyond knowledge-based pattern-based approaches, recent work consider question generation as a supervised machine learning task where questions or question patterns are generated by an end-to-end neural network directly from text (Dong et al, 2018;Duan et al, 2017) conditioned by answer spans, even considering jointly question generation and answer span identification . In (Du and Cardie, 2018), the SQuAD corpus is used to train a question generation model that first extract candidate answers from Wikipedia documents, then generate answer-specific questions.…”

Section: Related Workmentioning

confidence: 99%

CALOR-QUEST : generating a training corpus for Machine Reading Comprehension models from shallow semantic annotations

Béchet¹,

Aloui²,

Charlet³

et al. 2019

Proceedings of the 2nd Workshop on Machine Reading for Question Answering

View full text Add to dashboard Cite

Machine reading comprehension is a task related to Question-Answering where questions are not generic in scope but are related to a particular document. Recently very large corpora (SQuAD, MS MARCO) containing triplets (document, question, answer) were made available to the scientific community to develop supervised methods based on deep neural networks with promising results. These methods need very large training corpus to be efficient, however such kind of data only exists for English and Chinese at the moment. The aim of this study is the development of such resources for other languages by proposing to generate in a semi-automatic way questions from the semantic Frame analysis of large corpora. The collect of natural questions is reduced to a validation/test set. We applied this method on the French CALOR-FRAME corpus to develop the CALOR-QUEST resource presented in this paper.

show abstract

Neural Question Generation with Semantics of Question Type

Cited by 8 publications

References 13 publications

Evaluating the Knowledge Dependency of Questions

Evaluating the Knowledge Dependency of Questions

Review-based Question Generation with Adaptive Instance Transfer and Augmentation

CALOR-QUEST : generating a training corpus for Machine Reading Comprehension models from shallow semantic annotations

Contact Info

Product

Resources

About