SPECTRA: Sparse Structured Text Rationalization

Abstract

Selective rationalization aims to produce decisions along with rationales (e.g., text highlights or word alignments between two sentences). Commonly, rationales are modeled as stochastic binary masks, requiring samplingbased gradient estimators, which complicates training and requires careful hyperparameter tuning. Sparse attention mechanisms are a deterministic alternative, but they lack a way to regularize the rationale extraction (e.g., to control the sparsity of a text highlight or the number of alignments). In this paper, we present a unified framework for deterministic extraction of structured explanations via constrained inference on a factor graph, forming a differentiable layer. Our approach greatly eases training and rationale regularization, generally outperforming previous work on what comes to performance and plausibility of the extracted rationales. We further provide a comparative study of stochastic and deterministic methods for rationale extraction for classification and natural language inference tasks, jointly assessing their predictive power, quality of the explanations, and model variability.

Human-annotated rationales and explainable text classification: a survey

“…The form of the rationale plays a large role in choosing a suitable metric to calculate agreement. Rationales with different granularities should not be mixed: word-level rationales will probably not agree with sentence-level rationales, as such rationales have a different bandwidth (Guerreiro and Martins, 2021).…”

Investigating the Saliency of Sentiment Expressions in Aspect-Based Sentiment Analysis

“…To test the former type of system, McAuley et al ( 2012) created a small test set where each sentence in a review has been annotated with a particular aspect. These annotations serve as human/gold rationales in (Antognini and Faltings, 2021;Paranjape et al, 2020;Yu et al, 2021;Guerreiro and Martins, 2021). In our study, the original SemEval dataset already contains sentence-level aspect annotations and the human/gold rationales are the result of a further layer of annotation which highlights the sentiment expressions within these sentences.…”