When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion

Abstract

Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems. Firstly, standard metrics are not sensitive to improvements in consistency in document-level translations. Secondly, previous work on context-aware NMT assumed that the sentence-aligned parallel data consisted of complete documents while in most practical scenarios such document-level data constitutes only a fraction of the available parallel data. To address the first issue, we perform a human study on an English-Russian subtitles dataset and identify deixis, ellipsis and lexical cohesion as three main sources of inconsistency. We then create test sets targeting these phenomena. To address the second shortcoming, we consider a set-up in which a much larger amount of sentence-level data is available compared to that aligned at the document level. We introduce a model that is suitable for this scenario and demonstrate major gains over a context-agnostic baseline on our new benchmarks without sacrificing performance as measured with BLEU. 1 1. ambiguity arising when imperative and indicative verb forms are the same,

Context-Aware Monolingual Repair for Neural Machine Translation

“…Moreover, we notice that deixis scores are less sensitive to the amount of training data than lexical cohesion and ellipsis scores. The reason might be that, as we observed in our previous work (Voita et al, 2019), inconsistencies in translations due to the presence of deictic words and phrases are more frequent in this dataset than other types of inconsistencies. Also, as we show in Section 7, this is the phenomenon the model learns faster in training.…”

“…As a second baseline, we use the two-pass CADec model (Voita et al, 2019). The first pass produces sentence-level translations.…”

“…Closest to our work are two-pass models for document-level NMT (Xiong et al, 2019;Voita et al, 2019), where a second, context-aware model takes the translation and hidden representations of the sentence-level first-pass model as its input.…”

When and Why is Document-level Context Useful in Neural Machine Translation?

“…In this section, we review the existing documentlevel approaches for NMT and describe our strategies to filter out uninteresting words in the context input. We illustrate with an example of including one previous source sentence as the documentlevel context, which can be easily generalized also to other context inputs such as target hypotheses (Agrawal et al, 2018;Bawden et al, 2018;Voita et al, 2019) or decoder states (Tu et al, 2018;Maruf and Haffari, 2018;Miculicich et al, 2018).…”

Designing the Business Conversation Corpus

“…The omission of the pronouns occurs more frequently in spoken language than written language. Recently, context-aware translation models attract attention from many researchers (Tiedemann and Scherrer, 2017;Voita et al, 2018Voita et al, , 2019 to solve this kind of problem, however, there are almost no conversational parallel corpora with context information except noisy OpenSubtitles corpus.…”