Integrating Domain Terminology into Neural Machine Translation

Michon, Elise; Crego, Josep Maria; Sénellart, Jean

doi:10.18653/v1/2020.coling-main.348

Cited by 12 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous work has used sentence tags to convey domain (Kobus et al, 2017;Britz et al, 2017), speaker gender (Vanmassenhove et al, 2018), or target language formality (Sennrich et al, 2016a;Feely et al, 2019). Multiple tags can be used throughout source and target sentences, for example indicating linguistic features (Sennrich & Haddow, 2016;García-Martínez et al, 2016;Aharoni & Goldberg, 2017;Saunders et al, 2018) or custom terminology use (Dinu et al, 2019;Michon et al, 2020). Tags are usually incorporated throughout training, implicitly requiring the availability of reliable tags for large training datasets, although new tags can also be introduced during model adaptation .…”

Section: Tagsmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Fine-tuning on in-domain data allows good domain adaptation, but requires sufficient relevant bilingual data. Even if this is available, simple fine-tuning can cause overfitting to new data and catastrophic forgetting of previously learned behaviour. We survey approaches to domain adaptation for NMT, particularly where a system may need to translate across multiple domains. We divide techniques into those revolving around data selection or generation, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multidomain adaptation techniques to other lines of NMT research.

show abstract

Section: Tagsmentioning

confidence: 99%

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

Saunders¹

2022

jair

View full text Add to dashboard Cite

show abstract

“…In the development of the neural machine translation (NMT), significant studies proposed methods to integrate external specialized domain terminologies. It can be roughly divided into the following three categories [20]: 1). Placeholders.…”

Section: Domain Terminology Injectionmentioning

confidence: 99%

General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

Liu¹,

Cao²,

Geng³

et al. 2022

Preprint

View full text Add to dashboard Cite

The lack of label data is one of the significant bottlenecks for Chinese Spelling Check (CSC). Existing researches use the method of automatic generation by exploiting unlabeled data to expand the supervised corpus. However, there is a big gap between the real input scenario and automatic generated corpus. Thus, we develop a competitive general speller ECSpell which adopts the Error Consistent masking strategy to create data for pretraining. This error consistency masking strategy is used to specify the error types of automatically generated sentences which is consistent with real scene. The experimental result indicates our model outperforms previous state-of-the-art models on the general benchmark.Moreover, spellers often work within a particular domain in real life. Due to lots of uncommon domain terms, experiments on our built domain specific datasets show that general models perform terribly. Inspired by the common practice of input methods, we propose to add an alterable user dictionary to handle the zero-shot domain adaption problem. Specifically, we attach a User Dictionary guided inference module (UD) to a general token classification based speller. Our experiments demonstrate that ECSpell 𝑈 𝐷 , namely ECSpell combined with UD, surpasses all the other baselines largely, even approaching the performance on the general benchmark 1 . CCS Concepts: • Computing methodologies → Natural language processing.

show abstract

“…Thus, it can be argued that in-training approaches are inferior to the constrained decoding methods in terms of straightforward terminology integration and, indeed, Dinu et al (2019) report the terminology usage rate 6-9% less than the constrained decoding method. To ensure the appearance of terms in the output, Michon et al (2020) use placeholders with the help of morphosyntactic annotations. Even though the approach is effective for choosing a correctly inflected form, it depends on the availability and performance of morphological analysers both in source and target languages.…”

Section: Related Workmentioning

confidence: 99%