Findings of the 2019 Conference on Machine Translation (WMT19)

Barrault, Loïc; Bojar, Ondřej; Costa-jussà, Marta R.; Federmann, Christian; Fishel, Mark; Graham, Yvette; Haddow, Barry; Huck, Matthias; Koehn, Philipp; Malmasi, Shervin; Monz, Christof; Müller, Mathias; Pal, Santanu Kumar; Post, Matt; Zampieri, Marcos

doi:10.18653/v1/w19-5301

Cited by 336 publications

(303 citation statements)

References 118 publications

Supporting

Mentioning

301

Contrasting

Order By: Relevance

“…Our work empirically strengthens and extends the recommendations on human MT evaluation in previous work (Läubli et al, 2018;Toral, Castilho, et al, 2018), some of which have meanwhile been adopted by the large-scale evaluation campaign at WMT 2019 (Barrault et al, 2019): the new evaluation protocol uses original source texts only (R5) and gives raters access to document-level context (R2). The findings of WMT 2019 provide further evidence in support of our recommendations.…”

Section: Recommendationssupporting

confidence: 68%

A Set of Recommendations for Assessing Human–Machine Parity in Language Translation

Läubli

Castilho

Neubig

et al. 2020

jair

View full text Add to dashboard Cite

The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations. We reassess Hassan et al.'s 2018 investigation into Chinese to English news translation, showing that the finding of human-machine parity was owed to weaknesses in the evaluation design-which is currently considered best practice in the field. We show that the professional human translations contained significantly fewer errors, and that perceived quality in human evaluation depends on the choice of raters, the availability of linguistic context, and the creation of reference translations. Our results call for revisiting current best practices to assess strong machine translation systems in general and human-machine parity in particular, for which we offer a set of recommendations based on our empirical findings.

show abstract

Section: Recommendationssupporting

confidence: 68%

A Set of Recommendations for Assessing Human–Machine Parity in Language Translation

Läubli

Castilho

Neubig

et al. 2020

jair

View full text Add to dashboard Cite

show abstract

“…The official results of the competition are reported by WMT19 organizer (Barrault et al, 2019) and the same are presented in Table 4, 5, 6 and 7 respectively.…”

Section: Results and Analysismentioning

confidence: 97%

Neural Machine Translation: Hindi-Nepali

Laskar¹,

Bandyopadhyay²

2019

Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2)

View full text Add to dashboard Cite

With the extensive use of Machine Translation (MT) technology, there is progressively interest in directly translating between pairs of similar languages. Because the main challenge is to overcome the limitation of available parallel data to produce a precise MT output. Current work relies on the Neural Machine Translation (NMT) with attention mechanism for the similar language translation of WMT19 shared task in the context of Hindi-Nepali pair. The NMT systems trained the Hindi-Nepali parallel corpus and tested, analyzed in Hindi ⇔ Nepali translation. The official result declared at WMT19 shared task, which shows that our NMT system obtained Bilingual Evaluation Understudy (BLEU) score 24.6 for primary configuration in Nepali to Hindi translation. Also, we have achieved BLEU score 53.7 (Hindi to Nepali) and 49.1 (Nepali to Hindi) in contrastive system type.

show abstract

“…Since assessing the performance of documentlevel systems is one of the goals of WMT19 (Barrault et al, 2019), we decided to build NMT systems trained for translation of longer segments than single sentences. In this paper, we describe our five NMT systems submitted to WMT19 English→Czech news translation task (see Table 1).…”

Section: Introductionmentioning

confidence: 99%

English-Czech Systems in WMT19: Document-Level Transformer

Popel¹,

Macháček²,

Auersperger³

et al. 2019

Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)

Self Cite

View full text Add to dashboard Cite

We describe our NMT systems submitted to the WMT19 shared task in English→Czech news translation. Our systems are based on the Transformer model implemented in either Tensor2Tensor (T2T) or Marian framework.We aimed at improving the adequacy and coherence of translated documents by enlarging the context of the source and target. Instead of translating each sentence independently, we split the document into possibly overlapping multi-sentence segments. In case of the T2T implementation, this "documentlevel"-trained system achieves a +0.6 BLEU improvement (p < 0.05) relative to the same system applied on isolated sentences. To assess the potential effect document-level models might have on lexical coherence, we performed a semi-automatic analysis, which revealed only a few sentences improved in this aspect. Thus, we cannot draw any conclusions from this week evidence.

show abstract

Findings of the 2019 Conference on Machine Translation (WMT19)

Cited by 336 publications

References 118 publications

A Set of Recommendations for Assessing Human–Machine Parity in Language Translation

A Set of Recommendations for Assessing Human–Machine Parity in Language Translation

Neural Machine Translation: Hindi-Nepali

English-Czech Systems in WMT19: Document-Level Transformer

Contact Info

Product

Resources

About