A Morpheme-Based Weighting for Chinese-Mongolian Statistical Machine Translation

Yang, Zhenxin; Li, Miao; Chen, Lei; Sun, Kai

doi:10.1587/transinf.2016edl8080

Cited by 3 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…is paper cuts in from two aspects of the reranking instance extraction algorithm and feature selection, aiming to solve the problem of maximum entropy training data imbalance. In the experiment, the statistical machine translation system [13] based on the maximum entropy ordering model will be used as the baseline system.…”

Section: Statistical Machine Translation Based On the Maximum Entropy Phrase Reordering Modelmentioning

confidence: 99%

“…In Table 3, Dir represents a word posterior probability feature based on a fixed position, Win represents a word posterior probability feature based on a sliding window, sliding window t � 2, Lev represents a word posterior probability feature based on Levenshtein alignment. When aligning the 1-best translation hypothesis in the Nbest list with other translation hypotheses, the open-source toolkit TER [13] is used, and its "shift" function is turned off, which is WER alignment. e abovementioned three posterior probabilities have been discretized before use [10].…”

Section: Classification Experiments Based On Word Posteriormentioning

confidence: 99%

See 1 more Smart Citation

Model and Simulation of Maximum Entropy Phrase Reordering of English Text in Language Learning Machine

2020

Complexity

View full text Add to dashboard Cite

This paper proposes a feature extraction algorithm based on the maximum entropy phrase reordering model in statistical machine translation in language learning machines. The algorithm can extract more accurate phrase reordering information, especially the feature information of reversed phrases, which solves the problem of imbalance of feature data during maximum entropy training in the original algorithm, and improves the accuracy of phrase reordering in translation. In the experiment, they were combined with linguistic features such as parts of speech, words, and syntactic features extracted by using the syntax analyzer, and the maximum entropy classifier was used to predict translation errors, and the experimental verification was performed on the Chinese-English translation data set and compared. The experimental results show that different word posterior probabilities have a significant impact on the classification error rate, and the combination of linguistic features based on the word posterior probability can significantly reduce the classification error rate and improve the translation error prediction performance.

show abstract

Section: Statistical Machine Translation Based On the Maximum Entropy Phrase Reordering Modelmentioning

confidence: 99%

Section: Classification Experiments Based On Word Posteriormentioning

confidence: 99%

Model and Simulation of Maximum Entropy Phrase Reordering of English Text in Language Learning Machine

2020

Complexity

View full text Add to dashboard Cite

show abstract

“…The morphemes in a word indicate the basic word features and provide grammatical and semantic relations among words in the sentence. Mongolian morphological segmentation aims to split Mongolian words into their morphemes, which facilitates the Mongolian NLP tasks, such as name entity recognition (Wang et al, 2016;, information retrieval (Liu et al, 2012), machine translation (Fan et al, 2017;Yang et al, 2016), and speech synthesis (Liu et al, 2017). There are about 60 thousand of morphemes in Mongolian, and the number of their formed words is more than 7 million.…”

Section: Introductionmentioning

confidence: 99%

Incorporating Inner-word and Out-word Features for Mongolian Morphological Segmentation

Liu¹,

Su²,

Zhang³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Mongolian morphological segmentation is regarded as a crucial preprocessing step in many Mongolian related NLP applications and has received extensive attention. Recently, end-to-end segmentation approaches with long short-term memory networks (LSTM) have achieved excellent results. However, the inner-word features among characters in the word and the out-word features from context are not well utilized in the segmentation process. In this paper, we propose a neural network incorporating inner-word and out-word features for Mongolian morphological segmentation. The network consists of two encoders and one decoder. The inner-word encoder uses the self-attention mechanisms to capture the inner-word features of the target word. The out-word encoder employs a two layers BiLSTM network to extract out-word features in the sentence. Then, the decoder adopts a multi-head double attention layer to fuse the inner-word features and out-word features and produces the segmentation result. The evaluation experiment compares the proposed network with the baselines and explores the effectiveness of the sub-modules.

show abstract

“…Many researchers have investigated the use of Mongolian morphological information to solve the data sparsity. Yang et al [8] used the morphology in the factored translation modle for Chinese-Mongolian SMT. However, factored model has the shortcoming of computing expensive and the translation quality is easily affected by the generation model.…”

Section: Introductionmentioning

confidence: 99%

Constructing morpheme-based translation model for Mongolian-Chinese SMT

Yang

Chen

et al. 2015

2015 International Conference on Asian Language Processing (IALP)

View full text Add to dashboard Cite

The data sparsity and the morphological d ifference between Chinese and Mongolian are the main problems in Mongolian-Chinese statistical machine trans lation (SMT). In this paper, we propose a novel method to construct morpheme-based translation model by using Mongolian morpheme as the pivot language. First, we train Mongolian-Morpheme SMT and Morpheme-Chinese SMT system, achieving a balance in the morphology of the language pair. Then we construct a new phrase table via these two systems to enrich translation knowledge without any additional bilingual resources, which is suitable for low resource language pairs. Finally we incorporate this phrase table to our Mongolian-Chinese SMT system in two ways: by using multiple decoding paths and by the combination of two phrase tables. Experimental results demonstrate the effectiveness of our method with a maximum improvement of 1.37 BLEU points increment over baseline. Keywords-statistical machine translation; phrase table; pivot language; morpheme; multiple decoding paths; low resource language 978-1-4673-95 96-0/15/$31.00 ©2015 IEEE

show abstract

A Morpheme-Based Weighting for Chinese-Mongolian Statistical Machine Translation

Cited by 3 publications

References 8 publications

Model and Simulation of Maximum Entropy Phrase Reordering of English Text in Language Learning Machine

Model and Simulation of Maximum Entropy Phrase Reordering of English Text in Language Learning Machine

Incorporating Inner-word and Out-word Features for Mongolian Morphological Segmentation

Constructing morpheme-based translation model for Mongolian-Chinese SMT

Contact Info

Product

Resources

About