Arabic and English Relative Clauses and Machine Translation Challenges

Abstract

The study aims at performing an error analysis as well as providing an evaluation of the quality of neural machine translation (NMT) represented by Google Translate when translating relative clauses. The study uses two test suites are composed of sentences that contain relative clauses. The first test suite composes of 108 pair sentences that are translated from English to Arabic whereas the second composes of 72 Arabic sentences that are translated into English. Errors annotation is performed by 6 professional annotators. The study presents a list of the annotated errors divided into accuracy and fluency errors that occur based on MQM. Manual evaluation is also performed by the six professionals along with a BLEU automatic evaluation using the Tilde Me platform. The results show that fluency errors are more frequent than accuracy errors. They also show that the frequency of errors and MT quality when translating from English into Arabic is lower than the frequency of errors and MT quality when translating from Arabic into English is also presented. Based on the performed error analysis and both manual and automatic evaluation, it is pointed out that the gap between MT and professional human translation is still large.

Systematic Review of English/Arabic Machine Translation Postediting: Implications for AI Application in Translation Research and Pedagogy

“…Although the difference in the number of studies that researched Arabic-English MT compared to studies that researched English-Arabic translation is not highly significant, the results signal the need for conducting more studies that investigate MT in relation to Arabic-English. While some studies concluded that error frequency in Arabic-English translation is less than that in English-Arabic translation [57,58] stressed the need for postediting to improve MT adequacy and fluency in Arabic-English translation. The following chart provides a visual representation of results in terms of language-pair direction.…”

“…Evaluative studies stressed the significance of translation evaluation to improve the performance of Arabic MT engines and suggest strategies to overcome relevant shortcomings [55,61,65]. Considering the limited research on Arabic MT effectiveness in dealing with specialized content [100], some studies addressed MT shortcomings in translating specialized texts or textual components such as the legal discourse [100], literary features [66], proverbs [67], sentiment words [101], relative clauses [57], as well as social media vernacular [102]. Evaluative studies continued to dominate the literature on MT systems to highlight their affordances and limitations in terms of adequacy, accuracy, fluency, context sensitivity, terminology and other criteria and called for complementarity between MT and HT via pre-editing and postediting [50,53,58,63,70,103].…”