Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns

Abstract

The advent of context-aware NMT has resulted in promising improvements in the overall translation quality and specifically in the translation of discourse phenomena such as pronouns. Previous works have mainly focused on the use of past sentences as context with a focus on anaphora translation. In this work, we investigate the effect of future sentences as context by comparing the performance of a contextual NMT model trained with the future context to the one trained with the past context. Our experiments and evaluation, using generic and pronoun-focused automatic metrics, show that the use of future context not only achieves significant improvements over the context-agnostic Transformer, but also demonstrates comparable and in some cases improved performance over its counterpart trained on past context. We also perform an evaluation on a targeted cataphora test suite and report significant gains over the contextagnostic Transformer in terms of BLEU.

Exophoric Pronoun Resolution in Dialogues with Topic Regularization

“…Coreference resolution is the task of identifying coreference relations among different mentions. As a vital natural language understanding component, a good coreference system could benefit many downstream tasks such as machine translation (Guillou, 2012;Wong et al, 2020), dialog systems (Strube and Müller, 2003), question answering (Dasigi et al, 2019), andsummarization (Steinberger et al, 2007). Due to the weak semantic meaning of pronouns (Ehrlich, 1981), grounding pronouns to their referents (PCR) has been specially studied as a more challenging task than the general coreference resolution (Mitkov, 1998;Ng, 2005).…”

“…The coreference relationship between a pronoun and its referents is categorized into endophora and exophora based on whether the referred objects appear in text or out of text, and the former case can be further divided into anaphora if the referents appear in the preceding text of the pronoun and cataphora if in the following text (Halliday and Hasan, 1976;Brown and Yule, 1983). Conventional studies on the pronoun coreference resolution (PCR) task in the NLP community mainly focus on anaphora (Hobbs, 1978;NIST, 2003;Pradhan et al, 2012) and some recent work analyzes cataphora in machine translation (Wong et al, 2020), while mostly ignoring the exophoric pronouns. However, in daily dialogues or conversations, speakers may often use exophoric pronouns to refer to objects in the situational context that all speakers and listeners are aware of without introducing them in the first place.…”

Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models

“…A straightforward way to offset the sparsity of training signal is to increase the volume of training data. In fact, existing works that report strong results with targeted evaluation train their contextual parameters with millions of documentlevel sentence pairs (Bawden et al, 2018;Müller et al, 2018;Voita et al, 2019b;Zheng et al, 2020;Wong et al, 2020;Kang et al, 2020). In contrast, many works in the literature train models with the TED talks' subtitles released by the IWSLT shared tasks (Cettolo et al, 2012), which only consist of a couple of hundred thousand parallel sentences (see Table 1).…”

Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses

“…Most recent work on improving pronoun translations has involved building more complex architectures that incorporate contextual information (Voita et al, 2018;Wong et al, 2020). In contrast, we present a more generalized approach.…”