Log-linear Combinations of Monolingual and Bilingual Neural Machine
            Translation Models for Automatic Post-Editing

Junczys-Dowmunt, Marcin; Grundkiewicz, Roman

doi:10.18653/v1/w16-2378

Cited by 69 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model can be regarded as an automatic postediting system -a system designed to fix systematic MT errors that is decoupled from the main MT system. Automatic post-editing has a long history, including rule-based (Knight and Chander, 1994), statistical (Simard et al, 2007) and neural approaches (Junczys-Dowmunt and Grundkiewicz, 2016;Pal et al, 2016;Freitag et al, 2019). In terms of architectures, modern approaches use neural sequence-to-sequence models, either multi-source architectures that consider both the original source and the baseline translation (Junczys-Dowmunt and Grundkiewicz, 2016;Pal et al, 2016), or monolingual repair systems, as in Freitag et al (2019), which is concurrent work to ours.…”

Section: Automatic Post-editingmentioning

confidence: 98%

“…For training, the DocRepair model only requires monolingual document-level data. While we create synthetic training data via round-trip translation similarly to earlier work (Junczys-Dowmunt and Grundkiewicz, 2016;Freitag et al, 2019), note that we purposefully use sentence-level MT systems for this to create the types of consistency errors that we aim to fix with the context-aware DocRepair model. Not all types of consistency errors that we want to fix emerge from a round-trip translation, so access to parallel document-level data can be useful (Section 6.2).…”

Section: Automatic Post-editingmentioning

confidence: 99%

See 1 more Smart Citation

Context-Aware Monolingual Repair for Neural Machine Translation

Voita¹,

Sennrich²,

Titov³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Modern sentence-level NMT systems often produce plausible translations of isolated sentences. However, when put in context, these translations may end up being inconsistent with each other. We propose a monolingual DocRepair model to correct inconsistencies between sentence-level translations. DocRepair performs automatic post-editing on a sequence of sentence-level translations, refining translations of sentences in context of each other. For training, the DocRepair model requires only monolingual document-level data in the target language. It is trained as a monolingual sequence-to-sequence model that maps inconsistent groups of sentences into consistent ones. The consistent groups come from the original training data; the inconsistent groups are obtained by sampling roundtrip translations for each isolated sentence. We show that this approach successfully imitates inconsistencies we aim to fix: using contrastive evaluation, we show large improvements in the translation of several contextual phenomena in an English→Russian translation task, as well as improvements in the BLEU score. We also conduct a human evaluation and show a strong preference of the annotators to corrected translations over the baseline ones. Moreover, we analyze which discourse phenomena are hard to capture using monolingual data only. 1 1 The code and data sets (including round-trip translations) are available at https://github.com/lena-voita/ good-translation-wrong-in-context.

show abstract

Section: Automatic Post-editingmentioning

confidence: 98%

Section: Automatic Post-editingmentioning

confidence: 99%

Context-Aware Monolingual Repair for Neural Machine Translation

Voita¹,

Sennrich²,

Titov³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…In QE task, various methods are proposed, such as QuEst++, 10 which is a typical baseline using many features, a method that uses Gaussian Processes, 11 a method that predicts human post-edited sentences from original and translated sentences 12,13 and a Predictor-Estimator model [14][15][16] whose Predictor extracts features from source and translated sentences and Estimator estimates word tags and edit distances based on extracted features. Recently, many methods using pre-trained language models such as BERT and ELMo have been proposed.…”

Section: Using Post-edited Translations In Training a Quality Estimatmentioning

confidence: 99%

Estimating Machine Translation Quality of Any Input Sentence

Aida

Yamamoto

2020

Int. J. As. Lang. Proc.

View full text Add to dashboard Cite

Current methods of neural machine translation may generate sentences with different levels of quality. Methods for automatically evaluating translation output from machine translation can be broadly classified into two types: a method that uses human post-edited translations for training an evaluation model, and a method that uses a reference translation that is the correct answer during evaluation. On the one hand, it is difficult to prepare post-edited translations because it is necessary to tag each word in comparison with the original translated sentences. On the other hand, users who actually employ the machine translation system do not have a correct reference translation. Therefore, we propose a method that trains the evaluation model without using human post-edited sentences and in the test set, estimates the quality of output sentences without using reference translations. We define some indices and predict the quality of translations with a regression model. For the quality of the translated sentences, we employ the BLEU score calculated from the number of word [Formula: see text]-gram matches between the translated sentence and the reference translation. After that, we compute the correlation between quality scores predicted by our method and BLEU actually computed from references. According to the experimental results, the correlation with BLEU is the highest when XGBoost uses all the indices. Moreover, looking at each index, we find that the sentence log-likelihood and the model uncertainty, which are based on the joint probability of generating the translated sentence, are important in BLEU estimation.

show abstract

“…This means more careful decisions have to be made by the APE system, making the least possible edits to the raw mt. To this aim, we introduce our "conservativeness" penalty developed on the post editing penalty proposed by (Junczys-Dowmunt and Grundkiewicz, 2016). It is a simple yet effective method to penalize/reward hypotheses in the beam, at inference time, that diverge far from the original input.…”

Section: Conservativeness Penaltymentioning

confidence: 99%

Unbabel’s Submission to the WMT2019 APE Shared Task: BERT-Based Encoder-Decoder for Automatic Post-Editing

Lopes¹,

Farajian²,

Correia³

et al. 2019

Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2)

View full text Add to dashboard Cite

This paper describes Unbabel's submission to the WMT2019 APE Shared Task for the English-German language pair. Following the recent rise of large, powerful, pretrained models, we adapt the BERT pretrained model to perform Automatic Post-Editing in an encoder-decoder framework. Analogously to dual-encoder architectures we develop a BERT-based encoder-decoder (BED) model in which a single pretrained BERT encoder receives both the source src and machine translation mt strings. Furthermore, we explore a conservativeness factor to constrain the APE system to perform fewer edits. As the official results show, when trained on a weighted combination of in-domain and artificial training data, our BED system with the conservativeness penalty improves significantly the translations of a strong Neural Machine Translation (NMT) system by −0.78 and +1.23 in terms of TER and BLEU, respectively. Finally, our submission achieves a new state-of-the-art, exaequo, in English-German APE of NMT.

show abstract

Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

Cited by 69 publications

References 16 publications

Context-Aware Monolingual Repair for Neural Machine Translation

Context-Aware Monolingual Repair for Neural Machine Translation

Estimating Machine Translation Quality of Any Input Sentence

Unbabel’s Submission to the WMT2019 APE Shared Task: BERT-Based Encoder-Decoder for Automatic Post-Editing

Contact Info

Product

Resources

About