Inquisitive Question Generation for High Level Text Comprehension

Ko, Wei-Jen; Chen, Te-yuan; Huang, Yiyan; Durrett, Greg; Li, Junyi Jessy

doi:10.18653/v1/2020.emnlp-main.530

Cited by 25 publications

(49 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Minsky's frames (1975) embodied this idea, where he defined a frame as "a collection of questions to be asked about a hypothetical situation", and whose answers helped elaborate the situation to aid question-answering. Other studies have identified what questions people naturally ask when reading text (Ko et al, 2020) or viewing images (Mostafazadeh et al, 2016), to form a mental picture of what might be happening. Our work draws on these ideas to explore how coherent a LM's mental picture is, and how it can be improved.…”

Section: Related Workmentioning

confidence: 99%

DREAM: Improving Situational QA by First Elaborating the Situation

Gu¹,

Mishra²,

Clark³

2021

Preprint

View full text Add to dashboard Cite

To what extent do language models (LMs) build "mental models" of a scene when answering situated questions (e.g., questions about a specific ethical dilemma)? While cognitive science has shown that mental models play a fundamental role in human problemsolving, it is unclear whether the high questionanswering performance of existing LMs is backed by similar model building -and if not, whether that can explain their wellknown catastrophic failures. We observed that Macaw, an existing T5-based LM, when probed provides somewhat useful but inadequate mental models for situational questions (estimated accuracy=43%, usefulness=21%, consistency=42%). We propose DREAM, a model that takes a situational question as input to produce a mental model elaborating the situation, without any additional task specific training data for mental models. It inherits its social commonsense through distant supervision from existing NLP resources. Our analysis shows that DREAM can produce significantly better mental models (estimated accuracy=67%, usefulness=37%, con-sistency=71%) compared to Macaw. Finally, mental models generated by DREAM can be used as additional context for situational QA tasks. This additional context improves the answer accuracy of a Macaw zero-shot model by between +1% and +4% (absolute) on three different datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

DREAM: Improving Situational QA by First Elaborating the Situation

Gu¹,

Mishra²,

Clark³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Traditionally, QG is tackled by rule-based methods (Heilman and Smith, 2010;Labutov et al, 2015;Dhole and Manning, 2020) that rely heavily on extensive hand-crafted rules. Different from these, neural network-based methods are completely data-driven and trainable in an end-to-end fashion (Du et al, 2017;Song et al, 2018;Kim et al, 2018;Nema et al, 2019;Zhou et al, 2019a;Zhou et al, 2019b;Jia et al, 2020b;Ko et al, 2020). For better representing the input context, the answer position and token lexical features (e.g.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Question Generation with Commonsense Knowledge

Jia¹,

Wang²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Question generation (QG) is to generate natural and grammatical questions that can be answered by a specific answer for a given context. Previous sequence-to-sequence models suffer from a problem that asking high-quality questions requires commonsense knowledge as backgrounds, which in most cases can not be learned directly from training data, resulting in unsatisfactory questions deprived of knowledge. In this paper, we propose a multi-task learning framework to introduce commonsense knowledge into question generation process. We first retrieve relevant commonsense knowledge triples from mature databases and select triples with the conversion information from source context to question. Based on these informative knowledge triples, we design two auxiliary tasks to incorporate commonsense knowledge into the main QG model, where one task is Concept Relation Classification and the other is Tail Concept Generation. Experimental results on SQuAD show that our proposed methods are able to noticeably improve the QG performance on both automatic and human evaluation metrics, demonstrating that incorporating external commonsense knowledge with multi-task learning can help the model generate human-like and high-quality questions.

show abstract

“…Instead of asking factual questions with answers already present in the text, Ko et al (2020) are curiosity-driven, answer-agnostic, and seek a high-level understanding of the document being read. They released a dataset of such curiositydriven questions (henceforth INQUISITIVE; for details, see Section 2).…”

Section: Introductionmentioning

confidence: 99%

"What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation

Gao¹,

Ghosh²,

Gimpel³

2022

Preprint

View full text Add to dashboard Cite

We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and finetune models for type-controlled question generation. Empirical results demonstrate that we can generate a variety of questions that adhere to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs. inquisitive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses multiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitiveness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions. * Partially done as part of an internship at Educational Testing Service.

show abstract

Inquisitive Question Generation for High Level Text Comprehension

Cited by 25 publications

References 34 publications

DREAM: Improving Situational QA by First Elaborating the Situation

DREAM: Improving Situational QA by First Elaborating the Situation

Enhancing Question Generation with Commonsense Knowledge

"What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation

Contact Info

Product

Resources

About