Evaluating Explanation Methods for Neural Machine Translation

Abstract

Recently many efforts have been devoted to interpreting the black-box NMT models, but little progress has been made on metrics to evaluate explanation methods. Word Alignment Error Rate can be used as such a metric that matches human understanding, however, it can not measure explanation methods on those target words that are not aligned to any source word. This paper thereby makes an initial attempt to evaluate explanation methods from an alternative viewpoint. To this end, it proposes a principled metric based on fidelity in regard to the predictive behavior of the NMT model. As the exact computation for this metric is intractable, we employ an efficient approach as its approximation. On six standard translation tasks, we quantitatively evaluate several explanation methods in terms of the proposed metric and we reveal some valuable findings for these explanation methods in our experiments.

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

“…Preservation Check Giving the explanation (or data based on the explanation) as input to the predictive model should result in the same decision as for the original, full input sample. Feature importance, Heatmap, Localization, Text, Prototypes [23,34,40,60,87,88,122,134,147,148,159,215,276,292,297,298] Deletion Check Giving input without explanation's relevant features should result in a different decision by the predictive model than the decision for the original, full input sample. Feature importance, Heatmap, Localization [60,134,148,160,202,215] Fidelity Measure the agreement between the output of the predictive model and the explanation when applied to input sample(s).…”

“…[8,33,109]), or the number of decision rules in a set (e.g. [139,147,211,227,272,309]). Additionally, some evaluate the Redundancy of their explanations.…”

Measuring and Improving Faithfulness of Attention in Neural Machine Translation

“…While most of these works provide evidence that attention weights are not always faithful, Moradi et al (2019) confirm similar observations on the unfaithful nature of attention in the context of NMT models. Li et al (2020) is one of the few papers examining attention models in NMT. However, they are focused on the task of identifying relevant source words to explain the output translations selected by the NMT model.…”

“…This shows unfaithful behavior. (Samek et al, 2016;Mohseni and Ragan, 2018;Poerner et al, 2018;Jain and Wallace, 2019;Serrano and Smith, 2019;Wiegreffe and Pinter, 2019;Li et al, 2020).…”

TranSmart: A Practical Interactive Machine Translation System

“…Word Alignment Word alignment plays an important role in document translation for TranSmart. Since NMT is a blackbox model with massive parameters, previous work has made numerous efforts to induce word alignment from attention in NMT [53,54,55] or other explanation methods [56,57,58]. Other work improves alignment quality by building a word alignment model whose architecture is similar to NMT [59].…”