Data augmentation using back-translation for context-aware neural machine translation

Abstract

A single sentence does not always convey information required to translate it into other languages; we sometimes need to add or specialize words that are omitted or ambiguous in the source languages (e.g., zero pronouns in translating Japanese to English or epicene pronouns in translating English to French). To translate such ambiguous sentences, we exploit contexts around the source sentence, and have so far explored context-aware neural machine translation (NMT). However, a large amount of parallel corpora is not easily available to train accurate context-aware NMT models. In this study, we first obtain large-scale pseudo parallel corpora by back-translating target-side monolingual corpora, and then investigate its impact on the translation performance of context-aware NMT models. We evaluate NMT models trained with small parallel corpora and the large-scale pseudo parallel corpora on IWSLT2017 English-Japanese and English-French datasets, and demonstrate the large impact of the data augmentation for context-aware NMT models in terms of BLEU score and specialized test sets on ja→en 1 and fr→en. Back-translation Model en→ja Original Parallel Corpus Target-side Original Monolingual Corpus (en) Source-side Pseudo Monolingual Corpus (ja) Pseudo Parallel Corpus Train Backtranslate Context-aware Forward-translation Model ja→en

A review: preprocessing techniques and data augmentation for sentiment analysis

“…Data augmentation is a spotlight in recent years, from a limited training data will automatically generate more training data as considered semi-supervised learning. Sennrich et al [12], Sugiyama and Yoshinaga [13] used back translation technique to generate training data to improve performance of translation model. Fadaee at al.…”

“…The biggest disadvantage of these methods is not reserving meaning concerning the context of the sentences, so we present more complex approaches retaining the meaning as the original sentence. Back translation aims to obtain more training samples based on the translators, many research teams have used to improve translation models [12][13][14][15]23]. This technique is resolved by using the translators to translate the original data to a certain language, after that taking the translated data into the independent translator to translate back to the original language.…”

ruBTS: Russian Sentence Simplification Using Back-translation

“…One of the techniques to get pseudo parallel corpora for context-aware NMT models is data augmentation using back-translation [17]. So, taking this approach, we assume that sentence simplification can be partially solved with the back-translation technique without fine-tuning to a downstream task or training a new model.…”

Untitled

“…In computer vision, data augmentation technologies are widely applied to generate auxiliary training examples [20][21][22]. In NLP, back-translation has been proven to be effective in augmenting diverse instances [23][24][25]. We borrow this idea and translate training sentences into pivot languages.…”