Syntax-based Transformer for Neural Machine Translation

Ma, Chaoyong; Tamura, Akihiro; Utiyama, Masao; Sumita, Eiichiro; Zhao, Tiejun

doi:10.5715/jnlp.27.445

Cited by 5 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e study chose the International Spoken Language and its Translation Review Contest (IWSLT) 2019 data, which has a small data size, as the dataset for this experiment, including 220,000 Chinese-English parallel utterance pairs, pairs of test set data, and pairs of development data [26]. Since the LSTM attentional embedding-based English machine translation model could not be trained and learned directly on the IWSLT 2019 dataset, word vector transformation of the dataset was also required [27].…”

Section: Dataset Sources and Preprocessingmentioning

confidence: 99%

LSTM-Based Attentional Embedding for English Machine Translation

Jian

Xiang

2022

Scientific Programming

View full text Add to dashboard Cite

In order to reduce the workload of manual grading and improve the efficiency of grading, a computerized intelligent grading system for English translation based on natural language processing is designed. An attention-embedded LSTM English machine translation model is proposed. Firstly, according to the characteristics of the standard LSTM network model that uses fixed dimensional vectors to represent words in the encoding stage, an English machine translation model based on LSTM attention embedding is established; the structure level of the English translation scoring system is constructed. A linguistic model of the English translation scoring system is established, and the probability distribution of a particular sentence sequence or word sequence of the translated text is statistically calculated using the model. The results show that the English machine translation model based on LSTM attention embedding proposed in this study can enhance the representation of the source language contextual information and improve the performance of the English machine translation model and the quality of the translation compared with the English machine translation models constructed by existing neural network structures, such as standard LSTM models, RNN models, and GRU-Attention translation models.

show abstract

Section: Dataset Sources and Preprocessingmentioning

confidence: 99%

LSTM-Based Attentional Embedding for English Machine Translation

Jian

Xiang

2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…where y t , W w , and b w are attention mechanism layer parameters and a t represents the weight of the sequence input at the t-th time point in the whole input. erefore, through the attention mechanism layer, the input vector v t with a weight expression can be obtained; the calculation formula is shown in (6).…”

Section: A Topic Segmentation Model Based On Model Transfermentioning

confidence: 99%

“…Translation models can be divided into the following three categories according to different basic translation units and modelling methods: word-based translation, phrasebased translation, and syntax-based translation [6][7][8]. e word-based translation model takes the translated word pair as the basic translation unit.…”

Section: Introductionmentioning

confidence: 99%

Research on the Semantic Analysis Method of Translation Corpus Based on Natural Language Processing

Yue

2022

Scientific Programming

View full text Add to dashboard Cite

Accurate recognition and analysis of semantics is the most important research field in the process of English translation with the help of natural language processing technology. This paper proposes an English semantic analysis method based on the neural network. First, the idea of model transfer is used to construct a topic segmentation model and the topic granularity segmentation of the translated text is carried out. Then, in order to obtain all the information in the English text, the recursive neural network is selected to recognize the word model. In order to recognize English texts with different sentence patterns, the long-term and short-term memory network is selected to extract the useful information of the text. Through the experimental data measurement and analysis results, compared with the traditional sentence analysis methods, the accuracy of the proposed method is as high as 95.8% and the model occupies less hardware resources.

show abstract

DSISA: A New Neural Machine Translation Combining Dependency Weight and Neighbors

Li,

Zhang,

Luo

2024

ACM Trans. Asian Low-Resour. Lang. Inf. Process.

View full text Add to dashboard Cite

Most of the previous neural machine translations (NMT) rely on parallel corpus. Integrating explicitly prior syntactic structure information can improve the neural machine translation. In this paper, we propose a Syntax Induced Self-Attention (SISA) which explores the influence of dependence relation between words through the attention mechanism and fine-tunes the attention allocation of the sentence through the obtained dependency weight. We present a new model, Double Syntax Induced Self-Attention (DSISA), which fuses the features extracted by SISA and a compact convolution neural network (CNN). SISA can alleviate long dependency in sentence, while CNN captures the limited context based on neighbors. DSISA utilizes two different neural networks to extract different features for richer semantic representation and replaces the first layer of Transformer encoder. DSISA not only makes use of the global feature of tokens in sentences but also the local feature formed with adjacent tokens. Finally, we perform simulation experiments that verify the performance of the new model on standard corpora.

show abstract

Syntax-based Transformer for Neural Machine Translation

Cited by 5 publications

References 23 publications

LSTM-Based Attentional Embedding for English Machine Translation

LSTM-Based Attentional Embedding for English Machine Translation

Research on the Semantic Analysis Method of Translation Corpus Based on Natural Language Processing

DSISA: A New Neural Machine Translation Combining Dependency Weight and Neighbors

Contact Info

Product

Resources

About