Artificial Intelligent Decoding of Rare Words in Natural Language Translation Using Lexical Level Context

Abstract

Artificial intelligence based Machine Translation is a Natural Language Processing, attaining significant attention to fully automate the system that can translate source language content into the target languages. The proposed method is a Neural Machine Translation, end-to-end system, with the lexical level context for converting German sentences to English sentences. The Transformer based Machine Translation is used for translation for handling the occurrence of rare words in the lexical level context. The FSR group rare words together and represent sentence level context as Latent Topic Representation. WMT En-De bilingual parallel corpus is used for translation handling the Out of Vocabulary words using clustering of tags. In the existing method there is a mismatch of translation but the proposed system is more superior due to the sentence context inclusion. The model performance is enhanced with hyper parameter optimization obtaining a BLEU score with a better translation of source to target language. Finally minimizing the TER score to attain a better translation rate