Investigating Query Expansion and Coreference Resolution in Question Answering on BERT

Abstract

The Bidirectional Encoder Representations from Transformers (BERT) model produces state-of-the-art results in many question answering (QA) datasets, including the Stanford Question Answering Dataset (SQuAD). This paper presents a query expansion (QE) method that identifies good terms from input questions, extracts synonyms for the good terms using a widely-used language resource, WordNet, and selects the most relevant synonyms from the list of extracted synonyms. The paper also introduces a novel QE method that produces many alternative sequences for a given input question using same-language machine translation (MT). Furthermore, we use a coreference resolution (CR) technique to identify anaphors or cataphors in paragraphs and substitute them with the original referents. We found that the QA system with this simple CR technique significantly outperforms the BERT baseline in a QA task. We also found that our best-performing QA system is the one that applies these three preprocessing methods (two QE and CR methods) together to BERT, which produces an excellent F1 score (89.8 F1 points) in a QA task. Further, we present a comparative analysis on the performances of the BERT QA models taking a variety of criteria into account, and demonstrate our findings in the answer span prediction task.

Proceedings of the Fourth Workshop on Neural Generation and Translation

“…Same-language MT has been successfully used in many NLP applications, e.g. text-to-speech synthesis for creating alternative target sequences (Cahill et al, 2009), translation between varieties of the same language (Brazilian Portuguese to European Portuguese) (Fancellu et al, 2014), paraphrase generation (Plachouras et al, 2018), and producing many alternative sequences of a given input question in question answering (Bhattacharjee et al, 2020). In our case, we developed Portuguese-to-Portuguese MT systems that were able to generate n-best (same-language) alternative sentences of an input Portuguese sentence.…”

“…dcu (Haque et al, 2020) compared both phrasebased and neural models by extending the STAPLE data with additional corpora (selected for similarity to the task data under a language model), with the neural model performing better. They generate sets of high-scoring predictions according to beam searches, majority voting, and other techniques, and also run these initial translations through an additional paraphrasing model, placing third in the Portuguese track.…”

DWIE: An entity-centric dataset for multi-task document-level information extraction

“…Information Extraction (IE) plays a fundamental role as a backbone component in many downstream applications. For example, an application such as question answering may be improved by relying on relation extraction (RE) (Hu et al, 2019;Yu et al, 2017), coreference resolution (Bhattacharjee et al, 2020;Gao et al, 2019), named entity recognition (NER) (Molla et al, 2006;Singh et al, 2018), and entity linking (EL) (Broscheit, 2019;Chen et al, 2017) components. This also holds for other applications such as personalized news recommendation (Karimi et al, 2018;Wang et al, 2018Wang et al, , 2019, fact checking (Thorne & Vlachos, 2018;Zhang & Ghorbani, 2020), opinion mining (Sun et al, 2017), semantic search (Cifariello et al, 2019), and conversational agents (Roller et al, 2020).…”

Event and Entity Coreference using Trees to Encode Uncertainty in Joint Decisions

“…Coreference resolution refers to the task of detecting mentions of various entities and events and identifying groups of mentions referring to the same real-world entity or event. It is a fundamental NLP task that has several downstream applications such as question answering Bhattacharjee et al, 2020), textual entailment (Mitkov et al, 2012), building and maintaining KBs (Angeli et al, 2015;Angell et al, 2021), and multi-document summarization (Falke et al, 2017;Huang and Kurohashi, 2021). Often these downstream applications consume a set of documents, and thus require detection of coreference relations between event and entity mentions spread across documents such as multiple news articles.…”