Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Wang, Zhiguo; Ng, Patrick; Ma, Xiaofei; Nallapati, Ramesh; Xiang, Bing

doi:10.18653/v1/d19-1599

Cited by 188 publications

(139 citation statements)

References 20 publications

Supporting

Mentioning

137

Contrasting

Order By: Relevance

“…We follow the preprocessing approached proposed by Wang et al (2019) and split passages into 100-word long chunks with 50-word long strides. We use a BM25 retriever to retrieve the top n passages for each question as inputs to the reader and the Wikipedia dump provided by Chen et al ( 2017) as source corpus.…”

Section: Methodsmentioning

confidence: 99%

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

Riedel

Minervini

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Most approaches to Open-Domain Question Answering consist of a light-weight retriever that selects a set of candidate passages, and a computationally expensive reader that examines the passages to identify the correct answer. Previous works have shown that as the number of retrieved passages increases, so does the performance of the reader. However, they assume all retrieved passages are of equal importance and allocate the same amount of computation to them, leading to a substantial increase in computational cost. To reduce this cost, we propose the use of adaptive computation to control the computational budget allocated for the passages to be read. We first introduce a technique operating on individual passages in isolation which relies on anytime prediction and a per-layer estimation of an early exit probability. We then introduce SKY-LINEBUILDER, an approach for dynamically deciding on which passage to allocate computation at each step, based on a resource allocation policy trained via reinforcement learning. Our results on SQuAD-Open show that adaptive computation with global prioritisation improves over several strong static and adaptive methods, leading to a 4.3x reduction in computation while retaining 95% performance of the full model.

show abstract

Section: Methodsmentioning

confidence: 99%

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

Riedel

Minervini

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…BERT-Base currently supports 104 languages including Vietnamese. BERT is used in many different applications, especially in the question answering system [24] [25] [26]. We use the fine-tuned BERT model for multilingual Q&A created by Manuel Romero because this model has dataset that is compatible with original dataset SQuAD v1.1 but also extends more eleven languages including Vietnamese language [27].…”

Section: F Bert Modulementioning

confidence: 99%

BERT+vnKG: Using Deep Learning and Knowledge Graph to Improve Vietnamese Question Answering System

Phan¹,

Do²

2020

IJACSA

View full text Add to dashboard Cite

A question answering (QA) system based on natural language processing and deep learning is a prominent area and is being researched widely. The Long Short-Term Memory (LSTM) model that is a variety of Recurrent Neural Network (RNN) used to be popular in machine translation, and question answering system. However, that model still has certainly limited capabilities, so a new model named Bidirectional Encoder Representation from Transformer (BERT) emerged to solve these restrictions. BERT has more advanced features than LSTM and shows state-of-the-art results in many tasks, especially in multilingual question answering system over the past few years. Nevertheless, we tried applying multilingual BERT model for a Vietnamese QA system and found that BERT model still has certainly limitation in term of time and precision to return a Vietnamese answer. The purpose of this study is to propose a method that solved above restriction of multilingual BERT and applied for question answering system about tourism in Vietnam. Our method combined BERT and knowledge graph to enhance accurately and find quickly for an answer. We experimented our crafted QA data about Vietnam tourism on three models such as LSTM, BERT fine-tuned multilingual for QA (BERT for QA), and BERT+vnKG. As a result, our model outperformed two previous models in terms of accuracy and time. This research can also be applied to other fields such as finance, e-commerce, and so on.

show abstract

“…In this study, the BiLSTM-CRF model, known for its high performance, is used; the performance was improved by various feature extensions [21][22][23]. Recently, BERT has shown state-of-the-art performance in sequence labeling and classification NLP tasks [24].…”

Section: B Named Entity Recognition (Ner)mentioning

confidence: 99%

Named-Entity Recognition Using Automatic Construction of Training Data From Social Media Messaging Apps

Lee

2020

IEEE Access

View full text Add to dashboard Cite

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

Cited by 188 publications

References 20 publications

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering

BERT+vnKG: Using Deep Learning and Knowledge Graph to Improve Vietnamese Question Answering System

Named-Entity Recognition Using Automatic Construction of Training Data From Social Media Messaging Apps

Contact Info

Product

Resources

About