Tilde's Machine Translation Systems for WMT 2017

Pinnis, Mārcis; Krišlauks, Rihards; Miks, Toms; Deksne, Daiga; Sics, Valters

doi:10.18653/v1/w17-4737

Cited by 22 publications

(34 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous work also explored generative morphological models, known as Factored Translation Models, that explicitly integrate additional linguistic markup at the word level to learn morphology . In NMT training, using subword units such as byte-pair encoding (Sennrich, Haddow, and Birch 2016) has become a de facto standard in training competition grade systems Pinnis et al 2017). A few have tried morpheme-based segmentation (Bradbury and Socher 2016), and several even used character-based systems (Chung, Cho, and Bengio 2016;Lee, Cho, and Hofmann 2017) to achieve similar performance as the BPE-segmented systems.…”

Section: Morphologymentioning

confidence: 99%

On the Linguistic Representational Power of Neural Machine Translation Models

Belinkov

Durrani²,

Dalvi³

et al. 2020

Computational Linguistics

View full text Add to dashboard Cite

Despite the recent success of deep neural networks in natural language processing (NLP) and other spheres of artificial intelligence (AI), their interpretability remains a challenge. We analyze the representations learned by neural machine translation (NMT) models at various levels of granularity and evaluate their quality through relevant extrinsic properties. In particular, we seek answers to the following questions: (i) How accurately is word-structure captured within the learned representations, which is an important aspect in translating morphologically-rich languages? (ii) Do the representations capture long-range dependencies, and effectively handle syntactically divergent languages? (iii) Do the representations capture lexical semantics? We conduct a thorough investigation along several parameters: (i) Which layers in the architecture capture each of these linguistic phenomena; (ii) How does the choice of translation unit (word, character, or subword unit) impact the linguistic properties captured by the underlying representations? (iii) Do the encoder and decoder learn differently and independently? (iv) Do the representations learned by multilingual NMT models capture the same amount of linguistic information as their bilingual counterparts? Our data-driven, quantitative evaluation illuminates important aspects in NMT models and their ability to capture various linguistic phenomena. We show that deep NMT models trained in an end-to-end fashion, without being provided any direct supervision during the training process, learn a non-trivial amount of linguistic information. Notable findings include the following observations: i) Word morphology and part-of-speech information are captured at the lower layers of the model; (ii) In contrast, lexical semantics or non-local syntactic and semantic dependencies are better represented at the higher layers of the model; (iii) Representations learned using characters are more informed about wordmorphology compared to those learned using subword units; and (iv) Representations learned by multilingual models are richer compared to bilingual models.

show abstract

Section: Morphologymentioning

confidence: 99%

On the Linguistic Representational Power of Neural Machine Translation Models

Belinkov

Durrani²,

Dalvi³

et al. 2020

Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…The approximate size of each of the parallel corpora is 1.6M sentences. As a starting point, we use the data as pre-processed (filtered, normalised, tokenised) by Pinnis et al (2017b) for their experiments.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Subword Segmentation for Machine Translation Based on Grouping Words by Potential Roots

Zuters¹,

Strazds²

2019

BJMC

View full text Add to dashboard Cite

This paper proposes a new subword segmentation method for machine translation. The algorithm, which we call GenSeg, is generic in the sense that it can be applied to any language, but is designed with an emphasis on inflectional splitting, i.e. it attempts to split words on boundaries corresponding to inflectional suffixes. The main principle of the method is grouping together words that share a common middle substring, and then separating the best such substring from the rest of the word. GenSeg is a cross-language method extended with some language-specific morphological analysis rules (currently for the Latvian language). To verify its effectiveness, we performed machine translation experiments in two directions: Latvian-English and English-Latvian, obtaining minor improvements in translation quality when using our pre-processing method.

show abstract

“…In WMT 2017, Tilde participated with MLSTM-based NMT systems (Pinnis et al, 2017c). In this paper, we compare the MLSTMbased models with Transformer models for English-Estonian and Estonian-English and we show that the state-of-the-art of WMT 2017 is well behind the new models.…”

Section: Introductionmentioning

confidence: 87%

“…NMT models so far have struggled with translating rare or unseen words (not different surface forms, but rather different words) correctly (Pinnis et al, 2017c). Named entities and non-translatable entities (various product names, identifiers, etc.)…”

Section: Automatic Post-editing Of Named Entitiesmentioning

confidence: 99%

Tilde’s Machine Translation Systems for WMT 2018

Pinnis¹,

Rikters²,

Krišlauks³

2018

Proceedings of the Third Conference on Machine Translation: Shared Task Papers

Self Cite

View full text Add to dashboard Cite

The paper describes the development process of the Tilde's NMT systems that were submitted for the WMT 2018 shared task on news translation. We describe the data filtering and pre-processing workflows, the NMT system training architectures, and automatic evaluation results. For the WMT 2018 shared task, we submitted seven systems (both constrained and unconstrained) for English-Estonian and Estonian-English translation directions. The submitted systems were trained using Transformer models.

show abstract

Tilde's Machine Translation Systems for WMT 2017

Cited by 22 publications

References 9 publications

On the Linguistic Representational Power of Neural Machine Translation Models

On the Linguistic Representational Power of Neural Machine Translation Models

Subword Segmentation for Machine Translation Based on Grouping Words by Potential Roots

Tilde’s Machine Translation Systems for WMT 2018

Contact Info

Product

Resources

About