Sentence Embedding for Neural Machine Translation Domain Adaptation

Wang, Rui; Finch, Andrew; Utiyama, Masao; Sumita, Eiichiro

doi:10.18653/v1/p17-2089

Cited by 88 publications

(66 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, applying data weighting into NMT domain adaptation has attracted much attention. Wang et al (2017a) and Wang et al (2017b) proposed several sentence and domain weighting methods with a dynamic weight learning strategy. Zhang et al (2019a) ranked unlabeled domain training samples based on their similarity to in-domain data, and then adopts a probabilistic curriculum learning strategy during training.…”

Section: Related Workmentioning

confidence: 99%

Iterative Dual Domain Adaptation for Neural Machine Translation

Zeng¹,

Liu²,

Su³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Previous studies on the domain adaptation for neural machine translation (NMT) mainly focus on the one-pass transferring out-ofdomain translation knowledge to in-domain NMT model. In this paper, we argue that such a strategy fails to fully extract the domainshared translation knowledge, and repeatedly utilizing corpora of different domains can lead to better distillation of domain-shared translation knowledge. To this end, we propose an iterative dual domain adaptation framework for NMT. Specifically, we first pretrain in-domain and out-of-domain NMT models using their own training corpora respectively, and then iteratively perform bidirectional translation knowledge transfer (from indomain to out-of-domain and then vice versa) based on knowledge distillation until the indomain NMT model convergences. Furthermore, we extend the proposed framework to the scenario of multiple out-of-domain training corpora, where the above-mentioned transfer is performed sequentially between the indomain and each out-of-domain NMT models in the ascending order of their domain similarities. Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of our framework.

show abstract

Section: Related Workmentioning

confidence: 99%

Iterative Dual Domain Adaptation for Neural Machine Translation

Zeng¹,

Liu²,

Su³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…To model texts into vector space, the input tokens are represented as embeddings in deep learning models [28,29,30,45,46,55,57]. Previous work has shown that word representations in NLP tasks can benefit from character-level models, which aim at learning language representations directly from characters.…”

Section: Related Work a Augmented Embeddingmentioning

confidence: 99%

Effective Subword Segmentation for Text Comprehension

Zhang

Zhao

Ling

et al. 2019

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Representation learning is the foundation of machine reading comprehension and inference. In state-of-theart models, character-level representations have been broadly adopted to alleviate the problem of effectively representing rare or complex words. However, character itself is not a natural minimal linguistic unit for representation or word embedding composing due to ignoring the linguistic coherence of consecutive characters inside word. This paper presents a general subwordaugmented embedding framework for learning and composing computationally-derived subword-level representations. We survey a series of unsupervised segmentation methods for subword acquisition and different subword-augmented strategies for text understanding, showing that subword-augmented embedding significantly improves our baselines in various types of text understanding tasks on both English and Chinese benchmarks.

show abstract

“…Selecting NMT training samples similar to the in-domain data from the out-of-domain parallel corpus has been explored in [13]. The central idea of this work is to use both in-domain and out-of-domain parallel corpora to train an NMT system.…”

Section: Data Selection For Mt Trainingmentioning

confidence: 99%

“…Using each of these approaches, we generate sentence embedding vectors for in-domain (Fin) and out-of-domain (Fout) target side sentences. Along similar lines as [13], data selection is based on the relative distance δs of a sentence vector vs w.r.t. the in-domain and out-of-domain centroids CF in and CF out respectively, indicated by Eq.…”

Section: Data Selection For Mt Trainingmentioning

confidence: 99%

Language Model Bootstrapping Using Neural Machine Translation for Conversational Speech Recognition

Punjabi¹,

Arsikere²,

Garimella³

2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

Building conversational speech recognition systems for new languages is constrained by the availability of utterances capturing user-device interactions. Data collection is expensive and limited by speed of manual transcription. In order to address this, we advocate the use of neural machine translation as a data augmentation technique for bootstrapping language models. Machine translation (MT) offers a systematic way of incorporating collections from mature, resource-rich conversational systems that may be available for a different language. However, ingesting raw translations from a general purpose MT system may not be effective owing to the presence of named entities, intra sentential code-switching and the domain mismatch between the conversational data being translated and the parallel text used for MT training. To circumvent this, we explore following domain adaptation techniques: (a) sentence embedding based data selection for MT training, (b) model finetuning, and (c) rescoring and filtering translated hypotheses. Using Hindi language as the experimental testbed, we supplement transcribed collections with translated US English utterances. We observe a relative word error rate reduction of 7.8-15.6%, depending on the bootstrapping phase. Fine grained analysis reveals that translation particularly aids the interaction scenarios underrepresented in the transcribed data.

show abstract

Sentence Embedding for Neural Machine Translation Domain Adaptation

Cited by 88 publications

References 14 publications

Iterative Dual Domain Adaptation for Neural Machine Translation

Iterative Dual Domain Adaptation for Neural Machine Translation

Effective Subword Segmentation for Text Comprehension

Language Model Bootstrapping Using Neural Machine Translation for Conversational Speech Recognition

Contact Info

Product

Resources

About