Open Domain Question Answering over Tables via Dense Retrieval

Abstract

Recent advances in open-domain QA have led to strong models based on dense retrieval, but only focused on retrieving textual passages. In this work, we tackle open-domain QA over tables for the first time, and show that retrieval can be improved by a retriever designed to handle tabular context. We present an effective pre-training procedure for our retriever and improve retrieval quality with mined hard negatives. As relevant datasets are missing, we extract a subset of NATURAL QUESTIONS (Kwiatkowski et al., 2019) into a Table QA dataset. We find that our retriever improves retrieval results from 72.0 to 81.1 recall@10 and end-to-end QA results from 33.8 to 37.7 exact match, over a BERT based retriever.

Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models

“…As in Karpukhin et al's (2020) approach, this goal is achieved using hard-negatives retrieved from all the tables from the English Wikipedia dump as well as in-batch negatives. DTR outperforms BM25 by more than 40 percentage points on the NQ-TABLES dataset (Herzig et al, 2021). However, the experiments also show that TAPAS requires additional pre-training on the task of table retrieval on millions of tables scraped from Wikipedia.…”

“…There are four deep learning approaches for table retrieval (Shraga et al, 2020b;Pan et al, 2021;Chen et al, 2020b;Herzig et al, 2021). Shraga et al (2020b) treat tables and queries as multi-modal objects that consist of query, table caption, schema, rows and columns.…”

“…This allows to flatten the table by concatenating the rows to a one-dimensional sequence of tokens. Similar to Karpukhin et al (2020), Herzig et al (2021) make use of a bi-encoder approach. However, they use two TAPAS instances instead of BERT instances to encode the queries and the tables, respectively.…”

“…The training and evaluation of models examined in this paper on the task of multi-modal retrieval makes use of five datasets from related work: NQ (Kwiatkowski et al, 2019), NQ-TABLES (Herzig et al, 2021), WIKISQL (Zhong et al, 2017), a subset of WIKISQL, which we call WIKISQL ctx-independent , and OTT-QA (Chen et al, 2021). This section briefly explains the characteristics of these datasets and of our newly created multi-modal retrieval dataset comprising tables and texts, which we call MULTIMODALRETRIEVAL.…”

“…Given that BERT models only allow onedimensional strings of text as input, the twodimensional tables are transformed into one dimension by concatenating the titles of the page and the section the table occurs in, the caption of the table, and each row of the table. In order to analyze whether the table-specific language model TAPAS (Herzig et al, 2020; gives a performance boost compared to a plain BERT model, the remaining two bi-encoders make use of a TAPAS model that is pre-trained for the task of table retrieval (Herzig et al, 2021) for at least one of their encoders. Thus, the second bi-encoder uses a BERT-small instance as question encoder and a TAPAS-small instance as table and text encoder.…”