Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Abstract

We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER. Contrary to previous work, our method does not require access to any corpus-specific information, such as inter-document hyperlinks or human-annotated entity markers, and can be applied to any unstructured text corpus. Our system also yields a much better efficiency-accuracy trade-off, matching the best published accuracy on HotpotQA while being 10 times faster at inference time.

Recursive Decoding: A Situated Cognition Approach to Compositional Generation in Grounded Language Understanding

“…More broadly, RD parallels current trends in open-domain multi-hop question answering, where the state of art has moved to iteratively querying batches of source documents from a corpus (Xiong et al, 2020;Qi et al, 2020). While a different domain from grounded language understanding, this reflects an emerging recognition that a single forwardpass is often insufficient for complex, multi-stage reasoning or planning tasks.…”

More Robust Dense Retrieval with Contrastive Dual Learning

“…With the support of approximate nearest neighbor (ANN) search [11,15], dense retrievers can efficiently retrieve documents by conducting semantic matching in the embedding space. To encode queries and documents as dense representations, dense retrievers usually employ the BERT-Siamese architecture to provide fully learnable, well pretrained, and effective representations for queries and documents to construct embedding space for retrieval [16,40,41].…”

“…To optimize the embedding space for document retrieval, existing dense retrieval models usually sample negative documents and use them to contrastively train dense retrievers to learn query and document representations [16,40,41]. Wang et al [36] prove that contrastive representation learning optimizes neural models and keeps two properties of the document embedding space, "alignment" and "uniformity".…”

“…Wang et al [36] prove that contrastive representation learning optimizes neural models and keeps two properties of the document embedding space, "alignment" and "uniformity". The two properties, "alignment" and "uniformity", help to keep a homogeneous and isotropic embedding space [16,40,41] -which is that "alignment" assigns similar embedding features to query and its related documents and conducts better clustering for similar document representations; "uniformity" encourages encoders to maintain maximal information for documents. It helps to form a uniform document embedding space and better classifies the confusable documents according to queries.…”

“…It helps to form a uniform document embedding space and better classifies the confusable documents according to queries. The training object in existing dense retrieval methods [16,40,41] is to train dense retrievers on the document retrieval task and focuses on optimizing document embeddings. However, such document retrieval dominated training paradigm only optimizes the queries aligned to the relevant documents while discarding the uniformity nature in contrastive learning, leading to a non-smooth and non-uniform query embedding space.…”