Naver Labs Europe’s Systems for the WMT19 Machine Translation Robustness Task

Abstract

This paper describes the systems that we submitted to the WMT19 Machine Translation robustness task. This task aims to improve MT's robustness to noise found on social media, like informal language, spelling mistakes and other orthographic variations. The organizers provide parallel data extracted from a social media website 1 in two language pairs: French-English and Japanese-English (in both translation directions). The goal is to obtain the best scores on unseen test sets from the same source, according to automatic metrics (BLEU) and human evaluation. We proposed one single and one ensemble system for each translation direction. Our ensemble models ranked first in all language pairs, according to BLEU evaluation. We discuss the preprocessing choices that we made, and present our solutions for robustness to noise and domain adaptation.

Findings of the First Shared Task on Machine Translation Robustness

“…Data Cleaning Data cleaning played an important part in training successful MT systems in this campaign. Unlike other participants, the winning team Naver Labs Bérard et al (2019) and NTT (Murakami et al, 2019) applied data cleaning techniques in order to filter noisy parallel sentences. They filtered i) identical sentences on source and target side, ii) sentences that belonged to a language other than the source and target language, iii) sentences with length mismatch, and iv) also applied attention-based filtering.…”

“…The final submission is an ensemble of 4 models. NaverLabsEurope(NLE)' submission (Bérard et al, 2019): The participants carried substantial effort to clean the CommonCrawl data, applying length filtering (length ratio threshold), language identification-based filtering, and attention based filtering. They used the Transformer-Big architecture for Fra→Eng and Jpn→Eng, and Transformer-Base for the Eng→Jpn direction.…”

Monolingual Adapters for Zero-Shot Neural Machine Translation

“…Architecture We use the Transformer architecture (Vaswani et al, 2017), implemented in fairseq (Ott et al, 2019), which we modify to include monolingual and bilingual adapters. We train a joint BPE model (Sennrich et al, 2016) on all languages, with inline casing (Berard et al, 2019) and 64k merge operations (resulting in a 70k vocabulary size). The Transformer architecture used in this work 3 has 4 attention heads, 6 encoder layers, 6 decoder layers, an embedding size of 512 and a feed-forward dimension of 1024.…”

Adversarial Subword Regularization for Robust Neural Machine Translation

“…Michel and Neubig (2018) introduced a dataset scraped from Reddit for testing the NMT systems on the noisy text. Recently, a shared task on building the robust NMT models was held Bérard et al, 2019).…”