Rekha: A Reference Based Machine Translation Evaluation Metric Using Linguistic Knowledge and Contextual Embeddings

Abstract

Since the beginning of machine translation (MT) research, MT evaluation has been an area of interest of researchers. In literature, one can find more papers on MT evaluation than on machine translation itself. This paper describes the work done on developing our MT evaluation metric which incorporates linguistic as well as word embeddings for the evaluation of MT outputs. We have studied the performance of our metric on some English to Indian language machine translation systems. For this study, a comprehensive corpus was also developed which considered sentences based on different constructs. It was found that the proposed metric provides good results which are comparable with human (evaluation) judgments.