Reinforced Multi-task Approach for Multi-hop Question Generation

Abstract

Question generation (QG) attempts to solve the inverse of question answering (QA) problem by generating a natural language question given a document and an answer. While sequence to sequence neural models surpass rule-based systems for QG, they are limited in their capacity to focus on more than one supporting fact. For QG, we often require multiple supporting facts to generate high-quality questions. Inspired by recent works on multihop reasoning in QA, we take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context. We employ multitask learning with the auxiliary task of answer-aware supporting fact prediction to guide the question generator. In addition, we also proposed a question-aware reward function in a Reinforcement Learning (RL) framework to maximize the utilization of the supporting facts. We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA. Empirical evaluation shows our model to outperform the single-hop neural question generation models on both automatic evaluation metrics such as BLEU, METEOR, and ROUGE, and human evaluation metrics for quality and coverage of the generated questions.

Accelerating Real-Time Question Answering via Question Generation

“…We focus on text-based open-domain QA system, which first uses information retrieval (IR) to select passages, then applies a machine reading comprehension (MRC) model to extract answers from the passage pool. Since Chen et al (2017) built the first end-to-end system to tackle machine reading at scale, the IR phase has been investigated extensively (Wang et al 2018a;Kratzwald and Feuerriegel 2018;Lee et al 2018;Das et al 2019;Lee, Chang, and Toutanova 2019;Guu et al 2020). Meanwhile, MRC has also achieved great success in large-scale QA datasets, such as SQuAD (Rajpurkar et al 2016), especially after the advent of BERT (Devlin et al 2019).…”

“…Later on, some studies proposed to leverage direct or indirect information in paragraphs for QG (Du and Cardie 2018;Liu et al 2019a;Song et al 2018;Zhao et al 2018). To improve "shallow" questions that confine to a single sentence, deep QG has been explored (Pan et al 2020;Gupta et al 2020). More recently, large-scale language model pretraining has been applied to QG (Dong et al 2019;Bao et al 2020;Xiao et al 2020), which achieved state-of-the-art performance with significant improvement.…”