Evaluation of Rewriting Service in Language Translation Web Services Workflow

Abstract

We discuss here the complementarity effect of rewriting services in a language-translation Web service workflow. The communication mediated by the machine translation service includes mistranslations and changed meanings, which are caused by the quality of the machine translation. To reduce these problems, we propose service reallocation and assemblage of the rewriting service by humans at each stage of the workflow. We set up two rewriting service allocation patterns: 1) a reallocation pattern as a previous rewriting process, and 2) a reallocation pattern as a follow-on rewriting process, for workflows consisting only of machine translations. A team of human judges provide multiple assessments of adequacy and fluency of sample sentences that are translated from English to Japanese using each pattern. Results indicated that the Japanese rewriting task as a follow-on rewriting service provided greater fluency than an English rewriting task as a previous rewriting service, with nearly equal adequacy.

Mapping the Design Space of Human-AI Interaction in Text Summarization

“…Free-form text box: (Denkowski and Lavie, 2012;Yao et al, 2012;Yamaguchi et al, 2013;Turchi et al, 2014;Turner et al, 2015;Chu and Komlodi, 2017;Huang et al, 2020;Moramarco et al, 2021); Suggestion for edits (e.g., chatbot, substitution dropdowns): (Liu et al, 2011;Shi et al, 2013;Weisz et al, 2021;Passali et al, 2021); Scaffold for context (e.g., embedded dictionary): (Sugiyama et al, 2011;Lin, 2011); Productivity support (e.g., editing priority, auto-correction): (Lagarda et al, 2015;Peris and Casacuberta, 2019b;Wang et al, 2020;Zhao et al, 2020;Weisz et al, 2021) Interactive Editing…”

“…No Model Multiple choices: (Zhang et al, 2011;Pouliquen et al, 2011;Rosa et al, 2020;Stiennon et al, 2020); Rating (e.g., Likert scale, numerical, binary): (Nguyen et al, 2017;Lam et al, 2018Lam et al, , 2019Zarinbal et al, 2019;Bohn and Ling, 2021;Wu et al, 2021) Postediting Editing No Model Free-form text box: (Denkowski and Lavie, 2012;Yao et al, 2012;Yamaguchi et al, 2013;Turchi et al, 2014;Turner et al, 2015;Chu and Komlodi, 2017;Moramarco et al, 2021); Suggestion for edits (e.g., chatbot, substitution dropdowns): (Liu et al, 2011;Shi et al, 2013;Weisz et al, 2021;Passali et al, 2021); Scaffold for context (e.g., embedded dictionary): (Sugiyama et al, 2011;Lin, 2011); Productivity support (e.g., editing priority, auto-correction): (Lagarda et al, 2015;Peris and Casacuberta, 2019b;Wang et al, 2020;Weisz et al (Green et al, 2014;Torregrosa et al, 2014;Gero and Chilton, 2019;Santy et al, 2019;Buschek et al, 2021;Ippolito et al, 2019;Calderwood et al, 2020;Bhat et al, 2021;Clark and Smith, 2021); Substitution dropdowns: (Green et al, 2014;Torregrosa et al, 2014;…”