QuAC: Question Answering in Context

Abstract

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text. QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation. We also report results for a number of reference models, including a recently state-ofthe-art reading comprehension architecture extended to model dialog context. Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data. Dataset, baseline, and leaderboard available at http://quac.ai.

Question Answering for Privacy Policies: Combining Computational and Legal Perspectives

“…Our evaluation metric for answer-sentence selection is sentence-level F1, implemented similar to (Choi et al, 2018;Rajpurkar et al, 2016). Precision and recall are implemented by measuring the overlap between predicted sentences and sets of gold-reference sentences.…”

Attentive History Selection for Conversational Question Answering

“…In two recent ConvQA datasets, QuAC [2] and CoQA [27], Con-vQA is formalized as an answer span prediction problem similar in SQuAD [25,26]. Specifically, given a question, a passage, and the conversation history preceding the question, the task is to predict a span in the passage that answers the question.…”

“…On the aspect of history selection, existing models [2,27] select conversation history with a simple heuristic that assumes immediate previous turns are more helpful than others. This assumption, however, is not necessarily true.…”

A Split-and-Recombine Approach for Follow-up Query Analysis

“…Our work is closely related to two lines of work: context-dependent sentence analysis and reinforcement learning. From the perspective of context-dependent sentence analysis, our work is related to researches like reading comprehension in dialogue (Reddy et al, 2019;Choi et al, 2018), dialogue state tracking (Williams et al, 2013), conversational question answering in knowledge base (Saha et al, 2018;Guo et al, 2018), context-dependent logic forms (Long et al, 2016), and non-sentential utterance resolution in opendomain question answering (Raghu et al, 2015;Kumar and Joshi, 2017). The main difference is that we focus on the context-dependent queries in NLIDB which contain complex scenarios.…”