On Using Very Large Target Vocabulary for Neural Machine Translation

Jean, Sébastien; Cho, Kyunghyun; Memisevic, Roland; Bengio, Yoshua

doi:10.3115/v1/p15-1001

Cited by 644 publications

(591 citation statements)

References 15 publications

Supporting

Mentioning

581

Contrasting

Unclassified

Order By: Relevance

“…Performance is measured with BLEU (Papineni et al, 2002), and statistical significance is computed with bootstrap resampling (Koehn, 2004). The result of the word-level baseline system is computed after post-processing its output following the approach of Jean et al (2015), which was customized to our scenario. This method (see §2) is driven by the attention model to replace the UNK tokens in the output with their corresponding recommendation supplied as external knowledge.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Guiding Neural Machine Translation Decoding with External Knowledge

Chatterjee¹,

Negri²,

Turchi³

et al. 2017

Proceedings of the Second Conference on Machine Translation

View full text Add to dashboard Cite

Differently from the phrase-based paradigm, neural machine translation (NMT) operates on word and sentence representations in a continuous space. This makes the decoding process not only more difficult to interpret, but also harder to influence with external knowledge. For the latter problem, effective solutions like the XML-markup used by phrase-based models to inject fixed translation options as constraints at decoding time are not yet available. We propose a "guide" mechanism that enhances an existing NMT decoder with the ability to prioritize and adequately handle translation options presented in the form of XML annotations of source words. Positive results obtained in two different translation tasks indicate the effectiveness of our approach.

show abstract

Section: Resultsmentioning

confidence: 99%

“…A different technique is postprocessing the translated sentences. Jean et al (2015) and Luong and Manning (2015) replace the unknown words either with the most likely aligned source word or with the translation determined by another word alignment model.…”

Section: Related Workmentioning

confidence: 99%

Guiding Neural Machine Translation Decoding with External Knowledge

Chatterjee¹,

Negri²,

Turchi³

et al. 2017

Proceedings of the Second Conference on Machine Translation

View full text Add to dashboard Cite

show abstract

“…Kalchbrenner and Blunsom (2013) [5] used a standard RNN hidden unit for the decoder and a convolutional neural network for encoding the source sentence representation. However, at both the encoder and the decoder, Sutskever et al [9] all adopted a different version of the RNN with an LSTM-inspired hidden unit, the gated recur-rent unit (GRU), for both components. Bahdanau et al (2015) [8] successfully applied the attention mechanism into the NMT and proposed attention-based NMT to change the fixed vector c.…”

Section: A Rnn Encoder-decodermentioning

confidence: 99%

Key Research of Pre-processing on Mongolian-Chinese Neural Machine Translation

Hou

et al. 2016

Proceedings of the 2016 2nd International Conference on Artificial Intelligence and Industrial Engineering (AIIE 2016)

View full text Add to dashboard Cite

Abstract-Neural machine translation has recently achieved promising results with the big scale corpus. But there is little research on the small scale corpus, such as Mongolian. Mongolian belongs to the agglutinative language while Chinese is a pictograph. It is necessary to do some pre-processing for both Mongolian and Chinese before training the machine translation. In this paper, we successfully build an attention-based neural machine translation to do the CWMT2009 Mongolian to Chinese translation task. We also use four different approaches, respectively, to do the pre-processing for both Mongolian and Chinese, including segmenting Chinese into character, separating the Mongolian stem from the suffixes, addressing the case suffix and converting Mongolian into Latin. We carry out a lot of experiments to evaluate the approaches. We achieve the best BLEU with the score of 29.56. It is 1.82 points in BLEU score higher than the baseline which is trained with the original Mongolian and the general word segmentation of Chinese.

show abstract

“…Luong et al [14] and Li et al [12] propose simple alignment-based technique that can replace outof-vocabulary words with similar words. Jean et al [7] use a large vocabulary with a method based on importance sampling.…”

Section: Related Workmentioning

confidence: 99%

Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT

Wang

Zhang

et al. 2017

Communications in Computer and Information Science

View full text Add to dashboard Cite

Abstract. Neural machine translation (NMT), a new approach to machine translation, has been proved to outperform conventional statistical machine translation (SMT) across a variety of language pairs. Translation is an open-vocabulary problem, but most existing NMT systems operate with a fixed vocabulary, which causes the incapability of translating rare words. This problem can be alleviated by using different translation granularities, such as character, subword and hybrid word-character. Translation involving Chinese is one of the most difficult tasks in machine translation, however, to the best of our knowledge, there has not been any other work exploring which translation granularity is most suitable for Chinese in NMT. In this paper, we conduct an extensive comparison using Chinese-English NMT as a case study. Furthermore, we discuss the advantages and disadvantages of various translation granularities in detail. Our experiments show that subword model performs best for Chinese-to-English translation with the vocabulary which is not so big while hybrid word-character model is most suitable for Englishto-Chinese translation. Moreover, experiments of different granularities show that Hybrid BPE method can achieve best result on Chinese-toEnglish translation task.

show abstract

On Using Very Large Target Vocabulary for Neural Machine Translation

Abstract: arXiv:1412

Cited by 644 publications

References 15 publications

Guiding Neural Machine Translation Decoding with External Knowledge

Guiding Neural Machine Translation Decoding with External Knowledge

Key Research of Pre-processing on Mongolian-Chinese Neural Machine Translation

Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT

Contact Info

Product

Resources

About