Lexically Constrained Neural Machine Translation with Levenshtein Transformer

Abstract

This paper proposes a simple and effective algorithm for incorporating lexical constraints in neural machine translation. Previous work either required re-training existing models with the lexical constraints or incorporating them during beam search decoding with significantly higher computational overheads. Leveraging the flexibility and speed of a recently proposed Levenshtein Transformer model (Gu et al., 2019), our method injects terminology constraints at inference time without any impact on decoding speed. Our method does not require any modification to the training procedure and can be easily applied at runtime with custom dictionaries. Experiments on English-German WMT datasets show that our approach improves an unconstrained baseline and previous approaches.

EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints

“…It also drastically speeds up decoding compared with lexically constrained decoding algorithms (Post and Vilar, 2018). Furthermore, results highlight the benefits of soft constraints over hard ones-EDITOR with soft constraints achieves translation quality on par or better than both EDITOR and Levenshtein Transformer with hard constraints (Susanto et al, 2020).…”

“…• NAR models: EDITOR and LevT view the lexical constraints as soft constraints, provided via the initial target sequence. We also explore the decoding technique introduced in Susanto et al (2020) to support hard constraints.…”

“…Our work is closely related to Susanto et al (2020)'s idea of applying the Levenshtein Transformer to MT with hard terminology constraints. We will see that their technique can directly be used by EDITOR as well (Section 3.3), but this does not offer empirical benefits over the default EDITOR model (Section 4.3).…”

Integrating Domain Terminology into Neural Machine Translation

“…The approach generates all the constraints in the final output. Other works (Hasler et al, 2018;Post and Vilar, 2018;Susanto et al, 2020) attempt to reduce the computational problem caused by using multiple beams in the inference, a well known weakness of this approach. Similar to the previous approach, constrained decoding does not consider target context when inserting translation terms, as it sets the target form and then produces a target context that fits this constraint.…”

Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity

“…Incorporating bilingual lexicons and Phrases into NMT. Our method is also inspired by the studies of incorporating bilingual lexicons and phrases into NMT (Arthur et al, 2016;Zhang and Zong, 2016;Feng et al, 2017;Hasler et al, 2018;Zhao et al, 2018b;Zhao et al, 2018a;Dinu et al, 2019;Huck et al, 2019;Liu et al, 2019;Susanto et al, 2020). They utilize the external bilingual lexicons and phrases to improve the lexical and phrases translation.…”