R2-D2: A Modular Baseline for Open-Domain Question Answering

Fajcik, Martin; Docekal, Martin; Ondrej, Karel; Smrž, Pavel

doi:10.18653/v1/2021.findings-emnlp.73

Cited by 16 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They achieve competitive performance across different knowledge-intensive NLP tasks (Izacard and Grave, 2021b;Glass et al, 2021;Paranjape et al, 2021;Park et al, 2021;Borgeaud et al, 2021). Recent work improves the retrieval component (Paranjape et al, 2021;Maillard et al, 2021) or introduces another passage re-ranking modules (Fajcik et al, 2021) for further improvements. Our work focuses on improving the generator component, which has been underexplored in the literature.…”

Section: Related Workmentioning

confidence: 99%

“…This leverages a trained generator model and evaluates the relevance of each passage to a query through the correctness of the generated output when the passage is removed from the pool of retrieved passages. Unlike prior multi-task learning work in QA relying on available annotated data (Lee et al, 2021;Nishida et al, 2019) or heuristics such as answer string matching to label pseudo evidentiality (Fajcik et al, 2021), our approach is applicable to diverse downstream tasks, where we cannot use additional annotations or heuristics. Our evidentiality mining approach for high-quality silver labels can be applied to diverse NLP tasks, and our auxiliary task has a new purpose of evaluating passage evidentiality suitable for the open-retrieval.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

Asai¹,

Gardner²,

Hajishirzi³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open-domain question answering and fact verification. These models are trained to generate a final output given retrieved passages that can be irrelevant to an input query, leading to learning spurious cues or memorization. This work introduces a method to incorporate evidentiality of passages-whether a passage contains correct evidence to support the outputinto training the generator. We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage. Furthermore, we introduce a new task-agnostic method for obtaining high-quality silver evidentiality labels, addressing the issues of gold evidentiality labels being unavailable in most domains. Our experiments on five datasets across three knowledgeintensive tasks show that our new evidentialityguided generator significantly outperforms its direct counterpart on all of them, and advances the state of the art on three of them. Our analysis shows that the multi-task learning and silver evidentiality mining play key roles.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

Asai¹,

Gardner²,

Hajishirzi³

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Bi-encoder retrievers are effective in bringing out relevant passages from a large index but sometimes reranking those passages is essential as the downstream reader can only see a limited number of them. (Fajcik et al, 2021) uses reranker after their retriever as a cross-encoder to improve the recall which proves effective in the end-to-end question answering pipeline. Incorporating Wikidata (Hu et al, 2021) in translation for sentences including named entities is common in literature because the pretrained multilingual reader is unable to translate these entities as it has not seen them during training.…”

Section: Related Workmentioning

confidence: 99%

Zero-shot cross-lingual open domain question answering

Agarwal¹,

Tripathi²,

Mitamura³

et al. 2022

Proceedings of the Workshop on Multilingual Information Access (MIA)

View full text Add to dashboard Cite

People speaking different kinds of languages search for information in a cross-lingual manner. They tend to ask questions in their language and expect the answer to be in the same language, despite the evidence lying in another language. In this paper, we present our approach for this task of cross-lingual opendomain question-answering. Our proposed method employs a passage reranker, the fusionin-decoder technique for generation, and a wiki data entity-based post-processing system to tackle the inability to generate entities across all languages. Our end-2-end pipeline shows an improvement of 3 and 4.6 points on F1 and EM metrics respectively, when compared with the baseline CORA model on the XOR-TyDi dataset. We also evaluate the effectiveness of our proposed techniques in the zero-shot setting using the MKQA dataset and show an improvement of 5 points in F1 for high-resource and 3 points improvement for low-resource zero-shot languages. Our team, CMUmQA's submission in the MIA-Shared task ranked 1st in the constrained setup for the dev and 2nd in the test setting.

show abstract

“…This approach was also applied in RocketQA . UnitedQA (Cheng et al, 2021) and R2D2 (Fajcik et al, 2021) combine results from an ensemble of extractive and generative readers, whereas PAQ directly retrieves possible answers with an FiD fallback.…”

Section: E2e Optimization Of Nralmmentioning

confidence: 99%

You Only Need One Model for Open-domain Question Answering

Lee¹,

Kedia²,

Lee³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent works for Open-domain Question Answering refer to an external knowledge base using a retriever model, optionally rerank the passages with a separate reranker model and generate an answer using an another reader model. Despite performing related tasks, the models have separate parameters and are weakly-coupled during training. In this work, we propose casting the retriever and the reranker as hard-attention mechanisms applied sequentially within the transformer architecture and feeding the resulting computed representations to the reader. In this singular model architecture the hidden representations are progressively refined from the retriever to the reranker to the reader, which is more efficient use of model capacity and also leads to better gradient flow when we train it in an end-to-end manner. We also propose a pretraining methodology to effectively train this architecture. We evaluate our model on Natural Questions and TriviaQA open datasets and for a fixed parameter budget, our model outperforms the previous state-of-the-art model by 1.0 and 0.7 exact match scores.

show abstract

R2-D2: A Modular Baseline for Open-Domain Question Answering

Cited by 16 publications

References 12 publications

Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

Zero-shot cross-lingual open domain question answering

You Only Need One Model for Open-domain Question Answering

Contact Info

Product

Resources

About