Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering

Das, Rajarshi; Godbole, Ameya; Kavarthapu, Dilip; Gong, Zhiyu; Singhal, Abhishek; Yu, Mo; Guo, Xiaoxiao; Gao, Tian; Zamani, Hamed; Zaheer, Manzil; McCallum, Andrew

doi:10.18653/v1/d19-5816

Cited by 37 publications

(31 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Graph Representation of Documents. In general NLP research, people have built various text graphs by augmenting original text sequences with different hidden structural information, such as entitycentric graphs for efficient joint-encoding of large corpora (Wu et al, 2021;De Cao et al, 2019;Ding et al, 2019;Asai et al, 2020;Min et al, 2019;Das et al, 2019). Event graphs from a single document have been built for event schema induction (Li et al, 2018(Li et al, , 2020b, event coreference resolution (Phung et al, 2021;Zeng et al, 2021), etc.…”

Section: Related Workmentioning

confidence: 99%

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

Li¹,

Ma²,

Yu³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Timeline Summarization identifies major events from a news collection and describes them following temporal order, with key dates tagged. Previous methods generally generate summaries separately for each date after they determine the key dates of events. These methods overlook the events' intra-structures (arguments) and inter-structures (event-event connections). Following a different route, we propose to represent the news articles as an event-graph, thus the summarization task becomes compressing the whole graph to its salient sub-graph. The key hypothesis is that the events connected through shared arguments and temporal order depict the skeleton of a timeline, containing events that are semantically related, structurally salient, and temporally coherent in the global event graph. A time-aware optimal transport distance is then introduced for learning the compression model in an unsupervised manner. We show that our approach significantly improves the state of the art on three real-world datasets, including two public standard benchmarks and our newly collected Timeline 100 dataset. 1 Injure unconscious

show abstract

Section: Related Workmentioning

confidence: 99%

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

Li¹,

Ma²,

Yu³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multi-hop Reasoning: Many multifact reasoning approaches have been proposed for HotpotQA and similar datasets (Mihaylov et al, 2018;Khot et al, 2020). These use iterative fact selection (Nishida et al, 2019;Tu et al, 2020;Asai et al, 2020;Das et al, 2019), graph neural networks (Xiao et al, 2019;Fang et al, 2020;Tu et al, 2020), or simply cross-document self-attention (Yang et al, 2019;Beltagy et al, 2020) to capture inter-paragraph interaction. While these approaches have pushed the state of the art, the extent of actual progress on multifact reasoning remains unclear.…”

Section: Reducing Disconnected Reasoningmentioning

confidence: 99%

Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning

Trivedi

Balasubramanian

Khot³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Has there been real progress in multi-hop question-answering? Models often exploit dataset artifacts to produce correct answers, without connecting information across multiple supporting facts. This limits our ability to measure true progress and defeats the purpose of building multi-hop QA datasets. We make three contributions towards addressing this. First, we formalize such undesirable behavior as disconnected reasoning across subsets of supporting facts. This allows developing a model-agnostic probe for measuring how much any model can cheat via disconnected reasoning. Second, using a notion of contrastive support sufficiency, we introduce an automatic transformation of existing datasets that reduces the amount of disconnected reasoning. Third, our experiments 1 suggest that there hasn't been much progress in multifact QA in the reading comprehension setting. For a recent large-scale model (XLNet), we show that only 18 points out of its answer F1 score of 72 on HotpotQA are obtained through multifact reasoning, roughly the same as that of a simpler RNN baseline. Our transformation substantially reduces disconnected reasoning (19 points in answer F1). It is complementary to adversarial approaches, yielding further reductions in conjunction. * Early portion of this work was done during the first author's internship at Allen Institute for AI. 1 https://github.com/stonybrooknlp/dire Which country got independence when the cold war started? Supporting Facts (SF)

show abstract

“…We focus on the open-domain fullwiki Model Q/s Accuracy @2 @5 @10 @20 BM25 per second during inference on a single 16-core CPU. Accuracy @k is the fraction where both the correct passages are retrieved in the top k. † : Baselines obtained from Das et al (2019b). For DrKIT, we report the performance when the index is pretrained using the WikiData KB alone, the HotpotQA training questions alone, or using both.…”

Section: Hotpotqa: Multi-hop Information Retrievalmentioning

confidence: 99%

“…(Right) Overall performance on the HotpotQA task, when passing 10 retrieved passages to a downstream reading comprehension model (Yang et al, 2018). ‡ : From Das et al (2019b). : From Qi et al ( 2019).…”

Section: Hotpotqa: Multi-hop Information Retrievalmentioning

confidence: 99%

Differentiable Reasoning over a Virtual Knowledge Base

Dhingra¹,

Zaheer²,

Balachandran³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. At each step the module uses a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions. This module is differentiable, so the full system can be trained end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the contextual representation encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. On HotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking approach to retrieving the relevant passages required to answer a question. DrKIT is also very efficient, processing 10-100x more queries per second than existing multi-hop systems. 1

show abstract

Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering

Cited by 37 publications

References 24 publications

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport

Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning

Differentiable Reasoning over a Virtual Knowledge Base

Contact Info

Product

Resources

About