Towards User-Driven Neural Machine Translation

Lin, Huan; Liang, Yao; Yang, Baosong; Liu, Dayiheng; Zhang, Haibo; Luo, Weihua; Huang, Degen; Su, Jinsong

doi:10.18653/v1/2021.acl-long.310

Cited by 12 publications

(13 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pham et al (2019) change existing word embeddings to have domain-specific features which are activated or deactivated for a given input, improving lexical choice. Lin et al (2021) cache keywords for individual users with dedicated subnetworks, effectively tracking user 'domain'-specific vocabulary embeddings to combine with the generic embeddings. Sato et al (2020) replace generic translation word embeddings with domain-specific vocabulary embeddings learned by domain-specific language models from monolingual data.…”

Section: Domain-specific Subnetworkmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Fine-tuning on in-domain data allows good domain adaptation, but requires sufficient relevant bilingual data. Even if this is available, simple fine-tuning can cause overfitting to new data and catastrophic forgetting of previously learned behaviour. We survey approaches to domain adaptation for NMT, particularly where a system may need to translate across multiple domains. We divide techniques into those revolving around data selection or generation, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multidomain adaptation techniques to other lines of NMT research.

show abstract

Section: Domain-specific Subnetworkmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

show abstract

“…Contrastive Learning has been a widely-used technique to learn representations in both computer vision (Hjelm et al 2019;Khosla et al 2020;Chen et al 2020) and neural language processing (Logeswaran and Lee 2018;Fang et al 2020;Gao, Yao, and Chen 2021;Lin et al 2021). There are also several recent literatures that attempt to boost machine translation with the effectiveness of contrastive learning.…”

Section: Related Workmentioning

confidence: 99%

Frequency-Aware Contrastive Learning for Neural Machine Translation

Zhang

Yang

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

Low-frequency word prediction remains a challenge in modern neural machine translation (NMT) systems. Recent adaptive training methods promote the output of infrequent words by emphasizing their weights in the overall training objectives. Despite the improved recall of low-frequency words, their prediction precision is unexpectedly hindered by the adaptive objectives. Inspired by the observation that low-frequency words form a more compact embedding space, we tackle this challenge from a representation learning perspective. Specifically, we propose a frequency-aware token-level contrastive learning method, in which the hidden state of each decoding step is pushed away from the counterparts of other target words, in a soft contrastive way based on the corresponding word frequencies. We conduct experiments on widely used NIST Chinese-English and WMT14 English-German translation tasks. Empirical results show that our proposed methods can not only significantly improve the translation quality but also enhance lexical diversity and optimize word representation space. Further investigation reveals that, comparing with related adaptive training strategies, the superiority of our method on low-frequency word prediction lies in the robustness of token-level recall across different frequencies without sacrificing precision.

show abstract

“…Diverse NMT. Improving translation diversity has been a hot topic in NMT community in recent years, such as lattice-based NMT (Su et al, 2017;Tan et al, 2018) and personalized NMT (Michel and Neubig, 2018;Lin et al, 2021). Existing works for diverse NMT can be categorized into three major categories.…”

Section: Related Workmentioning

confidence: 99%

Bridging the Gap between Training and Inference: Multi-Candidate Optimization for Diverse Neural Machine Translation

Lin¹,

Yang²,

Yao³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

Self Cite

View full text Add to dashboard Cite

Diverse NMT aims at generating multiple diverse yet faithful translations given a source sentence. In this paper, we investigate a common shortcoming in existing diverse NMT studies: the model is usually trained with single reference, while expected to generate multiple candidate translations in inference. The discrepancy between training and inference enlarges the confidence variance and quality gap among candidate translations and thus hinders model performance. To deal with this defect, we propose a multi-candidate optimization framework for diverse NMT. Specifically, we define assessments to score the diversity and the quality of candidate translations during training, and optimize the diverse NMT model with two strategies based on reinforcement learning, namely hard constrained training and soft constrained training. We conduct experiments on NIST Chinese-English and WMT14 English-German translation tasks. The results illustrate that our framework is transparent to basic diverse NMT models, and universally makes better trade-off between diversity and quality. Our source code is available at https://github.com/ DeepLearnXMU/MultiCanOptim.

show abstract

Towards User-Driven Neural Machine Translation

Cited by 12 publications

References 27 publications

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Frequency-Aware Contrastive Learning for Neural Machine Translation

Bridging the Gap between Training and Inference: Multi-Candidate Optimization for Diverse Neural Machine Translation

Contact Info

Product

Resources

About