Korean-Vietnamese Neural Machine Translation with Named Entity Recognition and Part-of-Speech Tags

Abstract

Since deep learning was introduced, a series of achievements has been published in the field of automatic machine translation (MT). However, Korean-Vietnamese MT systems face many challenges because of a lack of data, multiple meanings of individual words, and grammatical diversity that depends on context. Therefore, the quality of Korean-Vietnamese MT systems is still sub-optimal. This paper discusses a method for applying Named Entity Recognition (NER) and Part-of-Speech (POS) tagging to Vietnamese sentences to improve the performance of Korean-Vietnamese MT systems. In terms of implementation, we used a tool to tag NER and POS in Vietnamese sentences. In addition, we had access to a Korean-Vietnamese parallel corpus with more than 450K paired sentences from our previous research paper. The experimental results indicate that tagging NER and POS in Vietnamese sentences can improve the quality of Korean-Vietnamese Neural MT (NMT) in terms of the BiLingual Evaluation Understudy (BLEU) and Translation Error Rate (TER) score. On average, our MT system improved by 1.21 BLEU points or 2.33 TER scores after applying both NER and POS tagging to the Vietnamese corpus. Due to the structural features of language, the MT systems in the Korean to Vietnamese direction always give better BLEU and TER results than translation machines in the reverse direction.

Human-machine Translation Model Evaluation Based on Artificial Intelligence Translation

“…(6) Repeat the above procedure until all predetermined system functions are confirmed. Once the SL sentences are put into the AI-based translation model, such operations of preprocessing as word and sentence segmentation and part-ofspeech tagging are required first to enable the AI-based translation to have a better understanding of the structures and syntactic relations of the input [24]. The attention mechanism is able to encode the context of the input words, sentences, sequences, and semantic information to change the preprocessed SL sentences into a continued vector representation.…”

AlphaBrains at WojoodNER shared task: Arabic Named Entity Recognition by Using Character-based Context-Sensitive Word Representations

“…These NEs are subsequently categorized into different semantic groups, such as names, places, organizations, events and dates, etc. NER is considered a crucial preliminary task for the development of different applications, such as, information retrieval (Popovski et al, 2020), text summarization (Khademi and Fakhredanesh, 2020), machine translation (Vu et al, 2020), topic modeling and event discovery (Feng et al, 2018), word-sense disambiguation (Al-Hajj and Jarrar, 2022) and others. NER is a typical sequence labeling token classification task where each token is assigned a tag.…”

Desenvolvimento e avaliação de um modelo NER no domínio da análise cultural e do turismo

“…O Reconhecimento de Entidades Mencionadas (NER, Named-Entity Recognition) é uma tarefa de extração de informação que consiste em identificar e classificar entidades tais como lugares, pessoas ou organizações. O NER é de grande utilidade para diversas tarefas em processamento de linguagem natural: por exemplo, em análise de sentimentos permite identificar a entidade sobre a que se emite uma opinião (Kanev et al, 2022;Barachi et al, 2022), e em tradução automática pode servir para selecionar aquelas entidades que não devem ser traduzidas (Vu et al, 2020;Lee et al, 2022). Tem sido aplicado a múltiplos domínios específicos, como saúde, segurança ou a extração de nomes em textos jornalísticos.…”