A Fine-grained Interpretability Evaluation Benchmark for Neural NLP

Abstract

While there is increasing concern about the interpretability of neural models, the evaluation of interpretability remains an open problem, due to the lack of proper evaluation datasets and metrics. In this paper, we present a novel benchmark to evaluate the interpretability of both neural models and saliency methods. This benchmark covers three representative NLP tasks: sentiment analysis, textual similarity and reading comprehension, each provided with both English and Chinese annotated data. In order to precisely evaluate the interpretability, we provide token-level rationales that are carefully annotated to be sufficient, compact and comprehensive. We also design a new metric, i.e., the consistency between the rationales before and after perturbations, to uniformly evaluate the interpretability of models and saliency methods on different tasks. Based on this benchmark, we conduct experiments on three typical models with three saliency methods, and unveil their strengths and weakness in terms of interpretability. We will release this benchmark at https://xyz and hope it can facilitate the research in building trustworthy systems.

Explainable Causal Analysis of Mental Health on Social Media Data

“…The IG is an attempt to assign an attribution value to each input feature which measures the extent to which an input contributes to the final prediction [12]. A recent study is carried out to set a benchmark over three representative NLP tasks (sentiment analysis, textual similarity and reading comprehension) for interpretability of both neural models and saliency methods [14] thereby emphasizing the need of LIME and IG for downstream NLP tasks.…”