Document Gated Reader for Open-Domain Question Answering

Abstract

In recent years researchers have achieved considerable success applying neural network methods to question answering (QA). These approaches have achieved state of the art results in simplified closed-domain settings 1 such as the SQuAD (Rajpurkar et al. 2016) dataset, which provides a preselected passage, from which the answer to a given question may be extracted. More recently, researchers have begun to tackle open-domain QA, in which the model is given a question and access to a large corpus (e.g., wikipedia) instead of a pre-selected passage (Chen et al. 2017a). This setting is more complex as it requires large-scale search for relevant passages by an information retrieval component, combined with a reading comprehension model that "reads" the passages to generate an answer to the question. Performance in this setting lags well behind closed-domain performance. In this paper, we present a novel open-domain QA system called Reinforced Ranker-Reader (R 3 ), based on two algorithmic innovations. First, we propose a new pipeline for open-domain QA with a Ranker component, which learns to rank retrieved passages in terms of likelihood of extracting the ground-truth answer to a given question. Second, we propose a novel method that jointly trains the Ranker along with an answer-extraction Reader model, based on reinforcement learning. We report extensive experimental results showing that our method significantly improves on the state of the art for multiple open-domain QA datasets. 2

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

“…DrQA (Chen et al, 2017) 37.7 44.5 41.9 48.7 32.3 38.3 29.8 -R 3 (Wang et al, 2018a) 35.3 41.7 49.0 55.3 47.3 53.7 29.1 37.5 OpenQA (Lin et al, 2018) 42.2 49.3 58.8 64.5 48.7 56.3 28.7 36.6 TraCRNet (Dehghani et al, 2019) 43.2 54.0 52.9 65.1 ----HAS-QA (Pang et al, 2019) 43.2 48.9 62.7 68.7 63.6 68.9 --BERT (Large) (Nogueira et al, 2018 sages with questions, aka inter-sentence matching (Wang and Jiang, 2017;Wang et al, 2016;Seo et al, 2017;Song et al, 2017). However, BERT model simply concatenates a passage with a question, and differentiates them by separating them with a delimiter token [SEP], and assigning different segment ids for them.…”

“…However, the question of proper granularity of passages is still underexplored. Third, passage ranker for selecting high-quality passages has been shown to be very useful in previous open-domain QA systems (Wang et al, 2018a;Lin et al, 2018;Pang et al, 2019). However, we do not know whether it is still required for BERT.…”

“…To answer these questions, we conduct a series of empirical studies on the OpenSQuAD dataset (Rajpurkar et al, 2016;Wang et al, 2018a). Experimental results show that: (1) global normalization makes QA model more stable while pinpointing answers from large number of passages;…”

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

“…Machine Reading at Scale First proposed and formalized in Chen et al (2017), MRS has gained popularity with increasing amount of work on both dataset collection (Joshi et al, 2017; and MRS model developments (Wang et al, 2018;Clark and Gardner, 2017;Htut et al, 2018). In some previous work , paragraph-level retrieval modules were mainly for improving the recall of required information, while in some other works (Yang et al, 2018), sentence-level retrieval modules were merely for solving the auxiliary sentence selection task.…”

Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering

“…The typical pipeline of open-domain QA systems (Chen et al, 2017;Wang et al, 2018;Htut et al, 2018) is to first use an IR system to retrieve a compact set of paragraphs and then run a machine reading model over the concatenated or reranked paragraphs. While IR works reasonably well for simple questions 1 , it often fails to retrieve the correct answer paragraph for multi-hop questions.…”