Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation

Yang, Yinfei; Jin, Ning; Lin, Kuo; Guo, Mandy; Cer, Daniel

doi:10.48550/arxiv.2009.13815

Cited by 9 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors show that this approach is able to improve the effectiveness of document retrieval. Similarly, [101] presents a method for question answering that uses a neural retrieval component and a cross-attention component to generate a response based on the retrieved passages. They also propose a data augmentation method to improve the performance of the model.…”

Section: Dense Retrieval Methodsmentioning

confidence: 99%

Information Retrieval: Recent Advances and Beyond

Hambarde¹,

Proença²

2023

Preprint

View full text Add to dashboard Cite

In this paper, we provide a detailed overview of the models used for information retrieval in the first and second stages of the typical processing chain. We discuss the current state-of-the-art models, including methods based on terms, semantic retrieval, and neural. Additionally, we delve into the key topics related to the learning process of these models. This way, this survey offers a comprehensive understanding of the field and is of interest for for researchers and practitioners entering/working in the information retrieval domain.

show abstract

Section: Dense Retrieval Methodsmentioning

confidence: 99%

Information Retrieval: Recent Advances and Beyond

Hambarde¹,

Proença²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Nie et al [93] proposed a decoupled encoding approach using DC-BERT, which enhances document retrieval effectiveness. Similarly, Yang et al [94] introduced a question-answering method that employs a neural retrieval component, a cross-attention mechanism for response generation, and a data augmentation technique to boost performance.…”

Section: ) Transformer-based Approachesmentioning

confidence: 99%

Information Retrieval: Recent Advances and Beyond

Hambarde

Proença

2023

IEEE Access

View full text Add to dashboard Cite

This paper provides an extensive and thorough overview of the models and techniques utilized in the first and second stages of the typical information retrieval processing chain. Our discussion encompasses the current state-of-the-art models, covering a wide range of methods and approaches in the field of information retrieval. We delve into the historical development of these models, analyze the key advancements and breakthroughs, and address the challenges and limitations faced by researchers and practitioners in the domain. By offering a comprehensive understanding of the field, this survey is a valuable resource for researchers, practitioners, and newcomers to the information retrieval domain, fostering knowledge growth, innovation, and the development of novel ideas and techniques.INDEX TERMS First-stage retrieval, information retrieval, second-stage retrieval.

show abstract

“…Oppositely, we focus on the more challenging one, which allows unanswerable questions and "non-contiguous" answer (Ravichander et al, 2021). In relevant literature works, retrieval augmented methods are applied in various contexts including privacy policies (e.g., Van et al (2021); Keymanesh et al (2021); Yang et al (2020)). Non-retrieval data aggregation has also been studied under different NLP contexts (e.g., bagging (Breiman, 1996), meta learning (Parvez et al, 2019)).…”

Section: Supplementary Material: Appendices a Related Workmentioning

confidence: 99%

Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies

Parvez,

Chi,

Ahmad

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Prior studies in privacy policies frame the question answering (QA) task as identifying the most relevant text segment or a list of sentences from a policy document given a user query. Existing labeled datasets are heavily imbalanced (only a few relevant segments), limiting the QA performance in this domain. In this paper, we develop a data augmentation framework based on ensembling retriever models that captures the relevant text segments from unlabeled policy documents and expand the positive examples in the training set. In addition, to improve the diversity and quality of the augmented data, we leverage multiple pre-trained language models (LMs) and cascade them with noise reduction filter models. Using our augmented data on the PrivacyQA benchmark, we elevate the existing baseline by a large margin (10% F1) and achieve a new state-of-the-art F1 score of 50%. Our ablation studies provide further insights into the effectiveness of our approach.

show abstract

Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation

Cited by 9 publications

References 21 publications

Information Retrieval: Recent Advances and Beyond

Information Retrieval: Recent Advances and Beyond

Information Retrieval: Recent Advances and Beyond

Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies

Contact Info

Product

Resources

About