S-Net: From Answer Extraction to Answer Synthesis for Machine Reading Comprehension

Abstract

In this paper, we present a novel approach to machine reading comprehension for the MS-MARCO dataset. Unlike the SQuAD dataset that aims to answer a question with exact text spans in a passage, the MS-MARCO dataset defines the task as answering a question from multiple passages and the words in the answer are not necessary in the passages. We therefore develop an extraction-then-synthesis framework to synthesize answers from extraction results. Specifically, the answer extraction model is first employed to predict the most important sub-spans from the passage as evidence, and the answer synthesis model takes the evidence as additional features along with the question and passage to further elaborate the final answers. We build the answer extraction model with state-of-the-art neural networks for single passage reading comprehension, and propose an additional task of passage ranking to help answer extraction in multiple passages. The answer synthesis model is based on the sequence-to-sequence neural networks with extracted evidences as features. Experiments show that our extraction-then-synthesis method outperforms state-of-the-art methods.

A Survey on Machine Reading Comprehension Systems

“…In our reviewed papers, 32% of works focus on non-factoid questions. Because of their difficulty, the systems dealing with non-factoid questions have often lower accuracies (Wang et al 2016;Tan et al 2018a;Wang et al 2018a).…”

“…In the abstractive mode, the answer is not necessarily an exact span in the context and is generated according to the question and context. This output type is especially suitable for non-factoid questions (Greco et al 2016;Tan et al 2018a).…”

“…One direction (Chen et al 2017), (Yang et al 2017b), (Weissenborn et al 2017), (Shen et al 2017), (Wang et al 2017b), (Huang et al 2018), (Tan et al 2018a), (Wang et al 2018b), (Hewlett et al 2017), (Xie and Xing 2017), (Salant and Berant 2018), (Wang et al 2018c), (Yadav, Vig and Shroff 2017), (Dhingra et al 2017), (Wang et al 2018a), (Kundu and Ng 2018b), (Prakash et al 2018), (Lin et al 2018), (Liu et al 2018d), (Liu et al 2018a), (Wang and Jiang 2017), (Chen et al 2016), (Kobayashi et al 2016), (Cui et al 2016), (Yin et al 2016), (Wang et al 2016), (Kadlec et al 2016a), (Kadlec et al 2016b), ), (Chaturvedi et al 2018), (Ghaeini et al 2018), (Sheng et al 2018, (Sugawara et al 2018), (Gupta et al 2018), (Ke et al 2018), (Tan et al 2018b), (Wang et al 2018f), (Mihaylov and Frank 2018), (Song et al 2018), (Sachan and Xing 2018), (Htut et al 2018), (Liu and Perez 2017),…”

Clozer”:" Adaptable Data Augmentation for Cloze-style Reading Comprehension

“…Answer extraction Tan et al (2018) develop an extraction-then-synthesis framework to synthesize answers from extraction results. Specifically, the answer extraction model is first employed to predict the most important sub-spans from the passage, then the answer synthesis model takes the subspans as additional features along with the question and passage to further elaborate the final answers.…”

Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts

“…However this might not be a good solution for MRC when there is a need of generating additional text not included in the passage or the question, augmenting information in multiple passage spans and the question as and when required. The work from (Weston et al, 2014), (Tan et al, 2017), (Cui et al, 2016) are examples of this change of paradigm. An interesting adaptation involves using single or multiple turns of reasoning to effectively exploit the relation among queries, documents, and answers.…”