Findings of the Third Workshop on Neural Generation and Translation

Hayashi, Hiroaki; Oda, Yusuke; Birch, Alexandra; Konstas, Ioannis; Finch, Andrew; Luong, Minh-Thang; Neubig, Graham; Sudoh, Katsuhito

doi:10.18653/v1/d19-5601

Cited by 17 publications

(28 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This paper describes the submissions of the "Marian" team to the Workshop on Neural Generation and Translation (WNGT 2019) efficiency shared task (Hayashi et al, 2019). The goal of the task is to build NMT systems on CPUs and GPUs placed on the Pareto Frontier of efficiency and accuracy.…”

Section: Introductionmentioning

confidence: 99%

From Research to Production and Back: Ludicrously Fast Neural Machine Translation

Kim¹,

Junczys-Dowmunt²,

Hassan³

et al. 2019

Proceedings of the 3rd Workshop on Neural Generation and Translation

View full text Add to dashboard Cite

This paper describes the submissions of the "Marian" team to the WNGT 2019 efficiency shared task. Taking our dominating submissions to the previous edition of the shared task as a starting point, we develop improved teacher-student training via multi-agent duallearning and noisy backward-forward translation for Transformer-based student models. For efficient CPU-based decoding, we propose pre-packed 8-bit matrix products, improved batched decoding, cache-friendly student architectures with parameter sharing and lightweight RNN-based decoder architectures. GPU-based decoding benefits from the same architecture changes, from pervasive 16bit inference and concurrent streams. These modifications together with profiler-based C++ code optimization allow us to push the Pareto frontier established during the 2018 edition towards 24x (CPU) and 14x (GPU) faster models at comparable or higher BLEU values. Our fastest CPU model is more than 4x faster than last year's fastest submission at more than 3 points higher BLEU. Our fastest GPU model at 1.5 seconds translation time is slightly faster than last year's fastest RNN-based submissions, but outperforms them by more than 4 BLEU and 10 BLEU points respectively.

show abstract

Section: Introductionmentioning

confidence: 99%

From Research to Production and Back: Ludicrously Fast Neural Machine Translation

Kim¹,

Junczys-Dowmunt²,

Hassan³

et al. 2019

Proceedings of the 3rd Workshop on Neural Generation and Translation

View full text Add to dashboard Cite

show abstract

“…The efficiency task complements machine translation quality evaluation campaigns by also measuring and optimizing the computational cost of inference. This is the third edition of the task, updating and building upon the second edition of the task (Hayashi et al, 2019).…”

Section: Efficiency Taskmentioning

confidence: 99%

Findings of the Fourth Workshop on Neural Generation and Translation

Hayashi¹,

Oda²,

Birch³

et al. 2020

Proceedings of the Fourth Workshop on Neural Generation and Translation

Self Cite

View full text Add to dashboard Cite

We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020). First, we summarize the research trends of papers presented in the proceedings. Second, we describe the results of the three shared tasks 1) efficient neural machine translation (NMT) where participants were tasked with creating NMT systems that are both accurate and efficient, and 2) document-level generation and translation (DGT) where participants were tasked with developing systems that generate summaries from structured data, potentially with assistance from text in another language and 3) STAPLE task: creation of as many possible translations of a given input text. This last shared task was organised by Duolingo.

show abstract

“…We use similar templates to generate a rough summary that is used as input in our rewrite model. Table 4: Generation results of our submitted systems as reported by the shared task organizers (Hayashi et al, 2019). RG: Relation Generation precision, CS: Content Selection (precision/recall), CO: Content Ordering.…”

Section: Generation With Pretrained Lmmentioning

confidence: 99%

Selecting, Planning, and Rewriting: A Modular Approach for Data-to-Document Generation and Translation

Miculicich

Marone²,

Hassan

2019

Proceedings of the 3rd Workshop on Neural Generation and Translation

View full text Add to dashboard Cite

In this paper, we report our system submissions to all 6 tracks of the WNGT 2019 shared task on Document-Level Generation and Translation. The objective is to generate a textual document from either structured data: generation task, or a document in a different language: translation task. For the translation task, we focused on adapting a large scale system trained on WMT data by fine tuning it on the RotoWire data. For the generation task, we participated with two systems based on a selection and planning model followed by (a) a simple language model generation, and (b) a GPT-2 pre-trained language model approach. The selection and planning module chooses a subset of table records in order, and the language models produce text given such a subset.

show abstract

Findings of the Third Workshop on Neural Generation and Translation

Cited by 17 publications

References 23 publications

From Research to Production and Back: Ludicrously Fast Neural Machine Translation

From Research to Production and Back: Ludicrously Fast Neural Machine Translation

Findings of the Fourth Workshop on Neural Generation and Translation

Selecting, Planning, and Rewriting: A Modular Approach for Data-to-Document Generation and Translation

Contact Info

Product

Resources

About