Exploring the Planet of the APEs: a Comparative Study of State-of-the-art Methods for MT Automatic Post-Editing

Abstract

Downstream processing of machine translation (MT) output promises to be a solution to improve translation quality, especially when the MT system's internal decoding process is not accessible. Both rule-based and statistical automatic postediting (APE) methods have been proposed over the years, but with contrasting results. A missing aspect in previous evaluations is the assessment of different methods: i) under comparable conditions, and ii) on different language pairs featuring variable levels of MT quality. Focusing on statistical APE methods (more portable across languages), we propose the first systematic analysis of two approaches. To understand their potential, we compare them in the same conditions over six language pairs having English as source. Our results evidence consistent improvements on all language pairs, a relation between the extent of the gain and MT output quality, slight but statistically significant performance differences between the two methods, and their possible complementarity.

The FBK Participation in the WMT15 Automatic Post-editing Shared Task

“…Recently, Chatterjee et al (2015) showed a fair systematic comparison of these two approaches over multiple language pairs and revealed that inclusion of source information in the form of context-aware variant is useful to improve translation quality over standard monolingual translation approach. They also showed that using monolingual translation alignment to build context-aware APE helps to mitigate the sparsity issue at the level of word alignment and for this reasons, we use this configuration to implement APE-2 method.…”

Guiding Neural Machine Translation Decoding with External Knowledge

“…The goal of automatic post-editing (APE) is to correct errors in an MT-ed text. The problem is typically approached as a "monolingual translation" task, in which models are trained on parallel corpora containing (MT_output, MT_post-edit) pairs, with MT post-edits coming from humans (Simard et al, 2007;Chatterjee et al, 2015bChatterjee et al, , 2017. In their attempt to translate the entire input sentence, APE systems usually tend to over-correct the source words, i.e.…”

Online Automatic Post-Editing across Domains

“…de Souza et al, 2014;C. de Souza et al, 2015) and automatic post-editing (APE) systems (Chatterjee et al, 2015b;Chatterjee et al, 2015a;Chatterjee et al, 2016). The APE components explored in this paper should be capable not only to spot recurring MT errors, but also to correct them.…”

“…Thus, integrating an APE system inside the CAT framework can further improve the quality of the suggested segments, reduce the workload of human post-editors and increase the productivity of translation industries. In the last decade many studies on APE have shown that the quality of the machine translated text can be improved significantly by post-processing the translations with an APE system (Simard et al, 2007;Dugast et al, 2007;Terumasa, 2007;Pilevar, 2011;Béchara et al, 2011;Chatterjee et al, 2015b). These systems mainly follow the phrase-based machine translation approach where the MT outputs (with optionally the source sentence) are used as the source language corpus and the post-edits are used as the target language corpus.…”