The TALP-UPC System for the WMT Similar Language Task: Statistical vs Neural Machine Translation

Abstract

Although the problem of similar language translation has been an area of research interest for many years, yet it is still far from being solved. In this paper, we study the performance of two popular approaches: statistical and neural. We conclude that both methods yield similar results; however, the performance varies depending on the language pair. While the statistical approach outperforms the neural one by a difference of 6 BLEU points for the Spanish-Portuguese language pair, the proposed neural model surpasses the statistical one by a difference of 2 BLEU points for Czech-Polish. In the former case, the language similarity (based on perplexity) is much higher than in the latter case. Additionally, we report negative results for the system combination with back-translation.Our TALP-UPC system submission won 1st place for Czech→Polish and 2nd place for Spanish→Portuguese in the official evaluation of the 1st WMT Similar Language Translation task.

Findings of the 2019 Conference on Machine Translation (WMT19)

“…The UPC-TALP team (Biesialska et al, 2019) by the Universitat Politècnica de Catalunya submitted a Transformer (implemented with Fairseq ) for the Czechto-Polish task and a Phrase-based system (implemented with Moses ) for Spanish-to-Portuguese. They tested adding monolingual data to the NMT system by copying the same data on the source and target sides, with negative results.…”

“…For the future it is worth investigating why languages from the same family, like Czech-Polish have extremely low performance. Authors in (Biesialska et al, 2019), with the best perform- Gamallo et al (2017) and provide big language distances for Czech-Polish compared to Spanish-Portuguese.…”

Evaluation of English–Slovak Neural and Statistical Machine Translation

“…In this Section, we focus on papers dealing with a linguistic evaluation of the MT outputs of various MT systems. Biesialska et al [13] analyzed the performance of the statistical and neural approaches to machine translation. They compared phrase-based and neural-based MT systems and their combination.…”