Visualizing and Understanding Neural Machine Translation

Ding, Yanzhuo; Liu, Yang; Luan, Huanbo; Sun, Maosong

doi:10.18653/v1/p17-1106

Cited by 143 publications

(128 citation statements)

References 15 publications

Supporting

Mentioning

126

Contrasting

Unclassified

Order By: Relevance

“…Some approaches have only been evaluated using visual inspection (Ding et al, 2017;Li et al, 2016a). Goyal et al (2016) identified important words for a visual question answering system and informally evaluated their approach by analyzing the distribution among PoS tags (e.g., assuming that nouns are important).…”

Section: Related Workmentioning

confidence: 99%

Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Nguyen¹

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

153

158

View full text Add to dashboard Cite

Text classification models are becoming increasingly complex and opaque, however for many applications it is essential that the models are interpretable. Recently, a variety of approaches have been proposed for generating local explanations. While robust evaluations are needed to drive further progress, so far it is unclear which evaluation approaches are suitable. This paper is a first step towards more robust evaluations of local explanations. We evaluate a variety of local explanation approaches using automatic measures based on word deletion. Furthermore, we show that an evaluation using a crowdsourcing experiment correlates moderately with these automatic measures and that a variety of other factors also impact the human judgements.

show abstract

Section: Related Workmentioning

confidence: 99%

Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Nguyen¹

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

153

158

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, Li et al (2016) presented the only work that directly employs saliency methods to interpret NLP models. Most similar to our work in spirit, Ding et al (2017) used Layer-wise Relevance Propagation (LRP; Bach et al 2015), an interpretation method resembling saliency, to interpret the internal working mechanisms of RNN-based neural machine translation systems. Although conceptually LRP is also a good fit for word alignment interpretation, we have some concerns with the mathematical soundness of LRP when applied to attention models.…”

Section: Related Workmentioning

confidence: 99%

Saliency-driven Word Alignment Interpretation for Neural Machine Translation

Ding¹,

Xu²,

Koehn³

2019

Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)

View full text Add to dashboard Cite

Despite their original goal to jointly learn to align and translate, Neural Machine Translation (NMT) models, especially Transformer, are often perceived as not learning interpretable word alignments. In this paper, we show that NMT models do learn interpretable word alignments, which could only be revealed with proper interpretation methods. We propose a series of such methods that are model-agnostic, are able to be applied either offline or online, and do not require parameter update or architectural change. We show that under the force decoding setup, the alignments induced by our interpretation method are of better quality than fast-align for some systems, and when performing free decoding, they agree well with the alignments induced by automatic alignment tools.

show abstract

“…A general method to determine input space relevances based on a backward decomposition of the neural network prediction function is layer-wise relevance propagation (LRP) (Bach et al, 2015). It was originally proposed to explain feed-forward neural networks such as convolutional neural networks (Bach et al, 2015;Lapuschkin et al, 2016), and was recently extended to recurrent neural networks (Arras et al, 2017b;Ding et al, 2017;Arjona-Medina et al, 2018).…”

Section: Layer-wise Relevance Propagationmentioning

confidence: 99%

“…Thus, methods that use additional information, such as training data statistics, sampling, or are optimization-based (Ribeiro et al, 2016;Lundberg and Lee, 2017;Chen et al, 2018) are out of our scope. Among the methods we consider, we note that the method of Murdoch et al (2018) was not yet compared against Arras et al (2017b); Ding et al (2017); and that the method of Ding et al (2017) was validated only visually. Moreover, to the best of our knowledge, no recurrent neural network explanation method was tested so far on a toy problem where the ground truth rele-vance value is known.…”

Section: Introductionmentioning

confidence: 99%

Evaluating Recurrent Neural Network Explanations

Arras¹,

Osman²,

Müller³

et al. 2019

Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

Recently, several methods have been proposed to explain the predictions of recurrent neural networks (RNNs), in particular of LSTMs. The goal of these methods is to understand the network's decisions by assigning to each input variable, e.g., a word, a relevance indicating to which extent it contributed to a particular prediction. In previous works, some of these methods were not yet compared to one another, or were evaluated only qualitatively. We close this gap by systematically and quantitatively comparing these methods in different settings, namely (1) a toy arithmetic task which we use as a sanity check, (2) a five-class sentiment prediction of movie reviews, and besides (3) we explore the usefulness of word relevances to build sentence-level representations. Lastly, using the method that performed best in our experiments, we show how specific linguistic phenomena such as the negation in sentiment analysis reflect in terms of relevance patterns, and how the relevance visualization can help to understand the misclassification of individual samples.

show abstract

Visualizing and Understanding Neural Machine Translation

Cited by 143 publications

References 15 publications

Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Saliency-driven Word Alignment Interpretation for Neural Machine Translation

Evaluating Recurrent Neural Network Explanations

Contact Info

Product

Resources

About