Can Rationalization Improve Robustness?

Abstract

A growing line of work has investigated the development of neural NLP models that can produce rationales-subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales ("rationalizer") before making predictions ("predictor"), they have the potential to ignore noise or adversarially added text by simply masking it out of the generated rationale. To this end, we systematically generate various types of 'AddText' attacks for both token and sentence-level rationalization tasks and perform an extensive empirical evaluation of state-of-the-art rationale models across five different tasks. Our experiments reveal that rationale models show promise in improving robustness but struggle in certain scenarios-e.g., when the rationalizer is sensitive to position bias or lexical choices of the attack text. Further, leveraging human rationales as supervision does not always translate to better performance. Our study is a first step towards exploring the interplay between interpretability and robustness in the rationalize-then-predict framework. 1

Human-annotated rationales and explainable text classification: a survey

“…Furthermore, hARs can teach ML models "valid reasons" for a classification, reducing spurious ML model behavior (Mathew et al, 2021;Chen et al, 2022;Joshi et al, 2022) and improving out-of-domain (OOD) performance (Lu et al, 2022).…”

Understanding Large Language Models : Towards Rigorous and Targeted Interpretability Using Probing Classifiers and Self-Rationalisation

“…This may also improve the model's robustness, as it may reduce the dependence on cues from one dataset that do not generalise to other datasets. While similar ideas are also applied to attribution methods (Chen et al 2022), the freer form of natural language explanations may extend the benefits to more tasks.…”

Learning to Ignore Adversarial Attacks

“…Where our study differs from most previous work is in using feature feedback for adversarial rather than out-ofdomain robustness. A concurrent work by Chen et al (2022) uses rationalization to improve robustness. The proposed method is similar to our work, but we explore supervision with attack tokens and achieve stronger robustness to additive attacks.…”