A Nil-Aware Answer Extraction Framework for Question Answering

Abstract

Recently, there has been a surge of interest in reading comprehension-based (RC) question answering (QA). However, current approaches suffer from an impractical assumption that every question has a valid answer in the associated passage. A practical QA system must possess the ability to determine whether a valid answer exists in a given text passage. In this paper, we focus on developing QA systems that can extract an answer for a question if and only if the associated passage contains an answer. If the associated passage does not contain any valid answer, the QA system will correctly return Nil. We propose a nil-aware answer span extraction framework that is capable of returning Nil or a text span from the associated passage as an answer in a single step. We show that our proposed framework can be easily integrated with several recently proposed QA models developed for reading comprehension and can be trained in an endto-end fashion. Our proposed nil-aware answer extraction neural network decomposes pieces of evidence into relevant and irrelevant parts and then combines them to infer the existence of any answer. Experiments on the NewsQA dataset show that the integration of our proposed framework significantly outperforms several strong baseline systems that use pipeline or threshold-based approaches.

Controlling Risk of Web Question Answering

“…There has been little work addressing the predictive uncertainty issue of MRC. Note that there have been some studies tackling the unanswerable question (i.e., null answer) problem in MRC [20,23,34], which is different from the predictive uncertainty issue we discuss here.…”

“…A common solution is to add a null position to the (a) A correct answer for the query "who invented the telegraph". passage [23,38]. Some work also introduced an additional model to detect those unanswerable questions [20].…”

Introduction to neural network‐based question answering over knowledge graphs

“…For instance, Jia and Liang (2017) demonstrate a drop of 35–75% in F1 scores of 16 models for the reading comprehension task trained over SQuAD (Rajpurkar, Zhang, Lopyrev, & Liang, 2016), by adversarially adding another sentence to the input paragraph (from which the system has to select the relevant span, given the question). Following, a new version of the aforementioned data set was released comprising of unanswerable questions (Rajpurkar, Jia, & Liang, 2018), leading to more robust reading comprehension approaches (Hu et al, 2018; Kundu & Ng, 2018). To the best of our knowledge, there has not been any work quantifying or improving the robustness of KGQA models.…”

Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling

“…Recent work shows that it is possible to determine lack of evidence with greater confidence by explicitly modeling for it. Works of Zhong et al (2019) and Kundu and Ng (2018) demonstrate model designs with specialized deep learning architectures that encode evidence in the input and show significant improvement in identifying unanswerable questions. This paper first introduces a baseline that is based on a language model.…”