The efficacy of human post-editing for language translation

Green, Spence; Heer, Jeffrey; Manning, Christopher D.

doi:10.1145/2470654.2470718

Cited by 160 publications

(128 citation statements)

References 23 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…Pause data was obtained from PET key-log files by calculating the intervals between any keyboard or mouse activity that resulted in a change in the text (navigation events were disregarded). This is similar to the strategy followed by Carl and Kay (2011) and Green et al (2013). As per , only when these intervals were 300 ms long or more they were considered pauses.…”

Section: Pause-to-word Ratio (Pwr)mentioning

confidence: 84%

See 1 more Smart Citation

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Vieira

2016

Machine Translation

View full text Add to dashboard Cite

There has been growing interest of late in the cognitive effort required by post-editing of machine translation. Compared to number of editing operations, cognitive (or mental) effort is frequently considered a more decisive indicator of the overall effort expended by post-editors. Estimating cognitive effort is not straightforward, however. Previous studies often triangulate different measures to obtain a consensus, but little post-editing research to date has attempted to show how measures of cognitive effort relate to each other in a multivariate analysis. This paper addresses this by presenting an exploratory comparison of cognitive measures based on eye tracking, pauses, editing time, and subjective ratings collected in a post-editing task carried out by professional and non-professional participants. All measures correlated with each other, but a principal components analysis showed that the measures cluster together in different ways. In particular, measures that increase with task time alone behaved differently from the others, with higher mutual associations and higher reliability. Regarding differences between professional and non-professional participants, it was observed that subjective ratings were overall more strongly associated with objective measures in the case of professionals. Surprising findings from previous research based on pause ratio are discussed. The paper argues that a pause typology will benefit the study of pause lengths and cognitive effort in post-editing.

show abstract

Section: Pause-to-word Ratio (Pwr)mentioning

confidence: 84%

“…Green et al 2013;Plitt and Masselot 2010). However, post-editing MT is not beneficial on all occasions.…”

Section: Introductionmentioning

confidence: 99%

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Vieira

2016

Machine Translation

View full text Add to dashboard Cite

show abstract

“…However, based on the TAPs, results observed here suggest that a distinction between these phases can be made in post-editing as well. It should be noted that in Green, Heer, and Manning (2013) post-editing was carried out sentence by sentence, a setting similar to the one adopted in the eye-tracking task described above. In contexts of this kind, quick editing operations implemented in the short textual span of a sentence may indeed dispense with separate phases for gisting and/or revising the text in post-editing.…”

Section: Discussionmentioning

confidence: 99%

Cognitive effort and different task foci in post-editing of machine translation: A think-aloud study

Vieira¹

2017

Across Languages and Cultures

View full text Add to dashboard Cite

Post-editing of machine translation is gaining popularity as a solution to the ever-increasing demands placed on human translators. There has been a great deal of research in this area aimed at determining the feasibility of post-editing, and at predicting post-editing effort based on source text features and machine translation errors. However, considerably less is known about the mental workings of post-editing and post-editors' decision-making or, in particular, the relationship between post-editing effort and different mental processes. This paper investigates these issues by analysing data from a think-aloud study through the lens of eye movements and subjective ratings obtained in a separate task. The results show that mental processes associated with grammar and lexis are significantly associated with cognitive effort in post-editing. This association was not observed for other aspects of the task concerning, for example, discourse or the real-world use of the text. In addition, it was noted that lexical issues are linked to long sequences of thought processes. The paper shows that lexis plays a central role in post-editing, and argues that more emphasis should be placed on this issue in future research and in post-editor training.

show abstract

“…In the first step, workers click on one word or phrase that needed to be corrected. In the next step, a separate group of workers proposed correc-1 A variety of HCI and NLP studies have confirmed the efficacy of monolingual or bilingual individuals post-editing of machine translation output (Callison-Burch, 2005;Koehn, 2010;Green et al, 2013). Past NLP work has also examined automatic post-editing (Knight and Chander, 1994). tions to problematic regions that had been identified by multiple workers in the first pass.…”

Section: Related Workmentioning

confidence: 99%

Are Two Heads Better than One? Crowdsourced Translation via a Two-Step Collaboration of Non-Professional Translators and Editors

Yan

Gao

Pavlick

et al. 2014

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Crowdsourcing is a viable mechanism for creating training data for machine translation. It provides a low cost, fast turnaround way of processing large volumes of data. However, when compared to professional translation, naive collection of translations from non-professionals yields low-quality results. Careful quality control is necessary for crowdsourcing to work well. In this paper, we examine the challenges of a two-step collaboration process with translation and post-editing by non-professionals. We develop graphbased ranking models that automatically select the best output from multiple redundant versions of translations and edits, and improves translation quality closer to professionals.

show abstract

The efficacy of human post-editing for language translation

Cited by 160 publications

References 23 publications

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Cognitive effort and different task foci in post-editing of machine translation: A think-aloud study

Are Two Heads Better than One? Crowdsourced Translation via a Two-Step Collaboration of Non-Professional Translators and Editors

Contact Info

Product

Resources

About