Modeling Target-Side Inflection in Neural Machine Translation

Abstract

NMT systems have problems with large vocabulary sizes.Byte-pair encoding (BPE) is a popular approach to solving this problem, but while BPE allows the system to generate any target-side word, it does not enable effective generalization over the rich vocabulary in morphologically rich languages with strong inflectional phenomena. We introduce a simple approach to overcome this problem by training a system to produce the lemma of a word and its morphologically rich POS tag, which is then followed by a deterministic generation step. We apply this strategy for English-Czech and EnglishGerman translation scenarios, obtaining improvements in both settings. We furthermore show that the improvement is not due to only adding explicit morphological information.

Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting

“…Morphological generation of previously unencountered word forms is a crucial problem in many areas of natural language processing (NLP). High performance can lead to better systems for downstream tasks, e.g., machine translation (Tamchyna et al, 2017). Since existing lexicons have limited coverage, learning morphological inflection patterns from labeled data is an important mission and has recently been the subject of multiple shared tasks (Cotterell et al, 2016(Cotterell et al, , 2017a.…”

The University of Helsinki submissions to the WMT18 news task

“…The output of the model is converted into surface forms in a separate, deterministic post-processing step. A similar two-step approach has been found to improve English to Czech NMT (Tamchyna et al, 2017), probably due to alleviating data sparsity caused by morphological complexity. As Finnish is also a morphologically complex language, adapting this approach to Finnish should result in a similar improvement.…”

“…The annotation format used differs from the one in Tamchyna et al (2017) in several aspects, the most important of which is that the morphological tags are not complex, multicategory tags that are interleaved one-to-one with lemmas. Instead, each lemma token can be followed by zero or more morphological tags, each corresponding to a nondefault value in a single morphological category: The first lemma komissio is the only one without any morphological tags, the rest of the lemmas are trailed by one or more tags.…”

Neural Generation for Czech: Data and Baselines

“…Moreover, the generated morphological tags following slot placeholders can be used to limit the scope of possible surface forms during lexicalization (see Section 3.3). This approach is inspired by similar approaches in phrase-based MT (Bojar, 2007;Toutanova et al, 2008;Fraser, 2009) and was developed in parallel to recent similar experiments with two-step neural MT (Nadejde et al, 2017;Tamchyna et al, 2017). We compare the lemma-tag generation mode against the TGen default direct word-form generation mode in our experiments.…”