Towards automatically generating Questions under Discussion to link information and discourse structure

Abstract

Questions under Discussion (QUD; Roberts, 2012) are emerging as a conceptually fruitful approach to spelling out the connection between the information structure of a sentence and the nature of the discourse in which the sentence can function. To make this approach useful for analyzing authentic data, Riester, Brunetti & De Kuthy ( 2018) presented a discourse annotation framework based on explicit pragmatic principles for determining a QUD for every assertion in a text. De demonstrate that this supports more reliable discourse structure annotation, and Ziai and Meurers (2018) show that based on explicit questions, automatic focus annotation becomes feasible. But both approaches are based on manually specified questions.In this paper, we present an automatic question generation approach to partially automate QUD annotation by generating all potentially relevant questions for a given sentence. While transformation rules can concisely capture the typical question formation process, a rule-based approach is not sufficiently robust for authentic data. We therefore employ the transformation rules to generate a large set of sentence-question-answer triples and train a neural question generation model on them to obtain both systematic question type coverage and robustness.

Exploring neural question generation for formal pragmatics: Data set and model evaluation

“…The generated question answer pairs in the QUACC corpus all satisfy the requirement of question answer congruence and as shown in De Kuthy et al (2020), this data set is a good source for training and testing of question generation approaches. De Kuthy et al ( 2020), Kannan et al (2021) trained word, character and subword seq2seq models successfully generating questions that satisfy question answer congruence.…”

“…In the following, we will use this cleaned corpus to evaluate a number of models that have been shown to provide good results for the task of QG with QA congruence. Our goal is to show that our clean QUACC data set enables different types of neural models to produce questions of even higher quality compared to the numbers that were presented in De Kuthy et al (2020) and Kannan et al (2021) where various models were only trained and tested on the unclean QUACC.…”

“…It requires data sets that contain question-answer pairs with explicit question answer congruence. First approaches exploring question generation under the perspective of question answer congruence are presented in the work of De Kuthy et al ( 2020 ) and Kannan et al ( 2021 ). Based on a newly created data set several word-based, character, and subword seq2seq models are trained and tested that successfully generate questions satisfying question answer congruence, i.e., questions that can be answered with the sentences given in the input.…”

“…To address this lack of data, this paper introduces QUACC, the Question Answer Congruence Corpus, a corpus of 5.3 millions question-answer pairs obtained from a German newspaper corpus, designed explicitly for the task of QG with direct question answer congruence. A first version of this corpus was presented in De Kuthy et al ( 2020 ). While they focused on the quality of the neural question generation models, they did not further investigate the quality of the newly created data set.…”

“…While they focused on the quality of the neural question generation models, they did not further investigate the quality of the newly created data set. Since neural models are very sensitive to the quality of the data, some of the quality issues observed by De Kuthy et al ( 2020 ), such as generation of incorrect question words, seem to be related to the errors in the data set. We therefore developed method to clean the original QUACC data set which will be discussed in this article.…”