Evaluation of Machine Translation Errors in English and Iraqi Arabic

Abstract

Errors in machine translations of English-Iraqi Arabic dialogues were analyzed at two different points in the systems" development using HTER methods to identify errors and human annotations to refine TER annotations. Although the frequencies of errors in the more mature systems were lower, the proportions of error types exhibited little change. Results include high frequencies of pronoun errors in translations to English, high frequencies of subject person inflection in translations to Iraqi Arabic, similar frequencies of word order errors in both translation directions, and very low frequencies of polarity errors. The problems with many errors can be generalized as the need to insert lexemes not present in the source or vice versa, which includes errors in multi-word expressions. Discourse context will be required to resolve some problems with deictic elements like pronouns.

A linguistically motivated taxonomy for Machine Translation error analysis

“…A similar work is presented in Condon et al (2010), but with translations to and from English to Iraqi Arabic. Errors were annotated as "Deletions", "Insertions" or "Substitutions" for morphological classes and after they were assigned a type of error following a similar taxonomy to that proposed by Vilar et al (2006).…”

A conjoint analysis framework for evaluating user preferences in machine translation

“…However, they do not provide insight into different types of MT errors. More fine-grained analyses of individual MT errors often include manual or (semi-)automatic error annotation to gain insights into the strengths and weaknesses of MT engines (Vilar et al 2006; Condon et al 2010; Popovic and Ney 2011; Farrús et al 2012). …”

Machine translation for Arabic dialects (survey)

“…Several prototype systems were developed for military and medical screening domains to enable conversations with local foreign language speakers of Iraqi Arabic, Mandarin, Farsi, Pashto, and Thai. Some research was dedicated to evaluate MT scores of Iraqi Arabic and English translators such as (Condon et al, 2010) and (Condon et al, 2008). In the same context, IBM MASTOR (Gao et al, 2006), is a speech-to-speech translation system that translates spontaneous free-form speech in real-time on both laptop and hand-held PDAs for two language pairs, English-Mandarin Chinese, and English-Arabic dialect.…”