Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning

Abstract

Neural models command state-of-the-art performance across NLP tasks, including ones involving “reasoning”. Models claiming to reason about the evidence presented to them should attend to the correct parts of the input while avoiding spurious patterns therein, be self-consistent in their predictions across inputs, and be immune to biases derived from their pre-training in a nuanced, context- sensitive fashion. Do the prevalent *BERT- family of models do so? In this paper, we study this question using the problem of reasoning on tabular data. Tabular inputs are especially well-suited for the study—they admit systematic probes targeting the properties listed above. Our experiments demonstrate that a RoBERTa-based model, representative of the current state-of-the-art, fails at reasoning on the following counts: it (a) ignores relevant parts of the evidence, (b) is over- sensitive to annotation artifacts, and (c) relies on the knowledge encoded in the pre-trained language model rather than the evidence presented in its tabular inputs. Finally, through inoculation experiments, we show that fine- tuning the model on perturbed data does not help it overcome the above challenges.

Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data

“…One potential reason is memorization. Previous works (Petroni et al, 2019;Carlini et al, 2021;Ishihara, 2023) show that large pretrained language models store knowledge in their parameters, which they tend to retrieve instead of reasoning over the provided input (Gupta et al, 2022a). Hence, models can memorize common arithmetic operations they encounter during training and perform well on certain downstream tasks.…”

“…question-answering) to NLI (Jena et al, 2022). Unlike unstructured text data, tables have a natural structure that allows creating controlled experiments more easily (Gupta et al, 2022a). We drew inspiration from prior tabular probing approaches and extended them for automating probing of numerical tabular data.…”

Generation of Training Examples for Tabular Natural Language Inference

“…For example, for the edge going from Node 𝑒 1 .Name to Node 𝑒 2 .Name, we add "c1.Name <> c2.Name AND" -pending AND are removed at the end (line 17). The resulting query is obtained by concatenating the three clauses and returning it (lines [17][18]. Once a query is derived, its execution on 𝑐 gives a result table like the one in Table 2.…”

“…In terms of reasoning requirements, about 80% of the examples in Totto [40] have sentences describing the data with text that does not contain mathematical expressions, such as max, min, and count, or comparison across values. (iii) They contain bias and errors that may lead to incorrect learning in the target models [18].…”