Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions

Abstract

This work proposes an extensive analysis of the Transformer architecture in the Neural Machine Translation (NMT) setting. Focusing on the encoder-decoder attention mechanism, we prove that attention weights systematically make alignment errors by relying mainly on uninformative tokens from the source sequence. However, we observe that NMT models assign attention to these tokens to regulate the contribution in the prediction of the two contexts, the source and the prefix of the target sequence. We provide evidence about the influence of wrong alignments on the model behavior, demonstrating that the encoder-decoder attention mechanism is well suited as an interpretability method for NMT. Finally, based on our analysis, we propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weights.

Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation

“…To compute the relative contribution of source and target tokens to each prediction made by the system we used an embedding perturbation method [41]. Given a source sentence x and its translation y, the absolute source contribution C S (y j ) when producing the probability of the j-th token y j is defined as the variance of y j 's output probability across N random perturbations of the word embeddings of x.…”

“…Note that this approach involves replacing some words in targetlanguage sentences with other words randomly chosen from 19. We set N = 50 and λ = 0.01 [41]. For high-resource conditions, mean and standard deviation of the source influence obtained when translating in-domain test sets with the baseline system, four other DA reference systems, and MaTiLDA using different transformations and combinations of them.…”

Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer

“…former computes gradients with respect to the input token embeddings to measure how much a change in the input changes the output. However, there is a tension between finding a faithful explanation and observing human-like alignments, since one does not imply the other (Ferrando and Costa-jussà, 2021).…”

“…Nonetheless, they apply their method on average over a dataset, not for getting input attributions of a single prediction. Gradientbased methods have also been extended to the target prefix (Ferrando and Costa-jussà, 2021), although they do not quantify the relative contribution of source and target inputs.…”

From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer

“…In ARE-based methods, which rely on an underlying classifier to predict if a post is toxic or not, the classifier is trained on the training part of the fold (which contains only toxic posts, ignoring the toxic span annotations) and a randomly selected but not in toxicity detection. See alsoWiegreffe and Pinter (2019),Kobayashi et al (2020), Ferrando andCosta-jussà (2021) for a broader discussion of attention as an explainability mechanism.…”