Neural Machine Translation Decoding with Terminology Constraints

Abstract

Despite the impressive quality improvements yielded by neural machine translation (NMT) systems, controlling their translation output to adhere to user-provided terminology constraints remains an open problem. We describe our approach to constrained neural decoding based on finite-state machines and multistack decoding which supports target-side constraints as well as constraints with corresponding aligned input text spans. We demonstrate the performance of our framework on multiple translation tasks and motivate the need for constrained decoding with attentions as a means of reducing misplacement and duplication when translating user constraints.

CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling

“…In neural machine translation, for example, grid beam search (Hokamp and Liu 2017, GBS) makes use of 2dimensional beam search to seek sentences that satisfy the constraints, whereas constrained beam search (Anderson et al 2017, CBS) utilizes a finite-state machine to assist searching. Post and Vilar (2018) and Hasler et al (2018) further accelerate the search process. In machine translation, the search space is limited and highly conditioned on the source sentence.…”

“…proach (Mou et al 2015), which could only generate sentences with one keyword. Additionally, researchers propose grid beam search to incorporate constraints in machine translation (Post and Vilar 2018;Hasler et al 2018;Hokamp and Liu 2017). It works with the translation task because the source and target are mostly aligned and the candidate set of translations is small.…”

On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation

“…Post-processing the hypotheses is another possibility, but this comes with the downside that offline modification of the hypotheses happens out of context. A third possibility is to do constrained decoding (Hokamp and Liu, 2017;Chatterjee et al, 2017;Hasler et al, 2018;Post and Vilar, 2018). This does not require knowledge of the constraints at training time, and it also allows dynamic changes of the rest of the hypothe-sis when the constraints are activated.…”

“…We include up to 4 dictionary entries per sentence, and add reference translations only if they are not part of the baseline (i.e. unconstrained) translation, similar to (Hasler et al, 2018). Table (3) shows results for the dictionary suggestions task described in Section (7).…”

“…One simple solution is to use maximum attention weights to extract the alignment, but this can result in wrong alignments in the case where the maximum attention weight is not pointing to the word being translated. Such cases are not uncommon in NMT, making the use of attention weights as alignment replacement non-trivial (Chatterjee et al, 2017;Hasler et al, 2018). Alignment extraction is even less clear for transformer models (Vaswani et al, 2017), which currently produce state-of-the-art results.…”