Multi-way, multilingual neural machine translation

Fırat, Orhan; Cho, Kyunghyun; Sankaran, Baskaran; Vural, Fatoş T. Yarman; Bengio, Yoshua

doi:10.1016/j.csl.2016.10.006

Cited by 70 publications

(47 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This mechanism guides the neural network to learn the important levels of features within the entire recognition task. It has been applied well in text translation [48], text matching [49], speech recognition [50], and especially image recognition [51]. SE Block proposed by He et al [36] is a deep learning method, which is a prominent implementation of attention mechanism.…”

Section: Methodsmentioning

confidence: 99%

A New Deep Learning Algorithm for SAR Scene Classification Based on Spatial Statistical Modeling and Features Re-Calibration

Chen

Cui

et al. 2019

Sensors

View full text Add to dashboard Cite

Synthetic Aperture Radar (SAR) scene classification is challenging but widely applied, in which deep learning can play a pivotal role because of its hierarchical feature learning ability. In the paper, we propose a new scene classification framework, named Feature Recalibration Network with Multi-scale Spatial Features (FRN-MSF), to achieve high accuracy in SAR-based scene classification. First, a Multi-Scale Omnidirectional Gaussian Derivative Filter (MSOGDF) is constructed. Then, Multi-scale Spatial Features (MSF) of SAR scenes are generated by weighting MSOGDF, a Gray Level Gradient Co-occurrence Matrix (GLGCM) and Gabor transformation. These features were processed by the Feature Recalibration Network (FRN) to learn high-level features. In the network, the Depthwise Separable Convolution (DSC), Squeeze-and-Excitation (SE) Block and Convolution Neural Network (CNN) are integrated. Finally, these learned features will be classified by the Softmax function. Eleven types of SAR scenes obtained from four systems combining different bands and resolutions were trained and tested, and a mean accuracy of 98.18% was obtained. To validate the generality of FRN-MSF, five types of SAR scenes sampled from two additional large-scale Gaofen-3 and TerraSAR-X images were evaluated for classification. The mean accuracy of the five types reached 94.56%; while the mean accuracy for the same five types of the former tested 11 types of scene was 96%. The high accuracy indicates that the FRN-MSF is promising for SAR scene classification without losing generality.

show abstract

Section: Methodsmentioning

confidence: 99%

A New Deep Learning Algorithm for SAR Scene Classification Based on Spatial Statistical Modeling and Features Re-Calibration

Chen

Cui

et al. 2019

Sensors

View full text Add to dashboard Cite

show abstract

“…The strategy is while translating from ES to FR, the Spanish sentence is first translated into English using the ES-to-EN NMT model, and then from 'both' the original Spanish sentence and the MT-ed English sentence, into a French sentence using a "multi-way multilingual NMT" model. However, the two strategies do not perform well in the reports [5].…”

Section: Pivot Machine Translationmentioning

confidence: 95%

“…An NMT-based pivot translation method has been proposed [5]. The architecture used in its 'one-to-one' strategy is the same as the "sentence translation strategy" described in [22].…”

Section: Pivot Machine Translationmentioning

confidence: 99%

“…The only difference is that SMT models are replaced by NMT models. A single attention model is introduced to be shared across all language pairs, which enables the training of multi-way translation system in one NMT model [5]. Accordingly, the second strategy proposed in [5] is the use of 'many-to-one' translation in pivot MT.…”

Section: Pivot Machine Translationmentioning

confidence: 99%

“…A single attention model is introduced to be shared across all language pairs, which enables the training of multi-way translation system in one NMT model [5]. Accordingly, the second strategy proposed in [5] is the use of 'many-to-one' translation in pivot MT. The strategy is while translating from ES to FR, the Spanish sentence is first translated into English using the ES-to-EN NMT model, and then from 'both' the original Spanish sentence and the MT-ed English sentence, into a French sentence using a "multi-way multilingual NMT" model.…”

Section: Pivot Machine Translationmentioning

confidence: 99%

See 2 more Smart Citations

Pivot Machine Translation Using Chinese as Pivot Language

Liu

Silva²,

Wang

et al. 2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

Pivoting through a popular language with more parallel corpora available (e.g. English and Chinese) is a common approach to build machine translation (MT) systems for low-resource languages. For example, to build a Russian-to-Spanish MT system, we could build one system using the Russian-Spanish corpus directly. We could also build two systems, Russian-to-English and English-to-Spanish, as the resources of the two language pairs are much larger than the Russian-Spanish pair, and use them cascadingly to translate texts in Russian into Spanish by pivoting through English. There are, however, some confusing results on the Pivot MT approach in the literature. In this paper, we reviewed the performance of Pivot MT with the United Nations Parallel Corpus v1.0 (UN6Way) using both English and Chinese as pivot languages. We also report our system performance on the CWMT 2018 Pivot MT shared task, where Japanese patent sentences are translated into English using Chinese as the pivot language.

show abstract

From bilingual to multilingual neural‐based machine translation by incremental training

Escolano

Costa-jussà

Fonollosa

2020

Asso for Info Science & Tech

View full text Add to dashboard Cite

A common intermediate language representation in neural machine translation can be used to extend bilingual systems by incremental training. We propose a new architecture based on introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we can train multiple encoders and decoders for each language, sharing among them a common intermediate representation. Translation results on the low-resource tasks (Turkish-English and Kazakh-English tasks) show a BLEU improvement of up to 2.8 points. However, results on a larger dataset (Russian-English and Kazakh-English) show BLEU losses of a similar amount. While our system provides improvements only for the low-resource tasks in terms of translation quality, our system is capable of quickly deploying new language pairs without the need to retrain the rest of the system, which may be a game changer in some situations. Specifically, what is most relevant regarding our architecture is that it is capable of: reducing the number of production systems, with respect to the number of languages, from quadratic to linear; incrementally adding a new language to the system without retraining the languages already there; and allowing for translations from the new language to all the others present in the system.

show abstract

Multi-way, multilingual neural machine translation

Cited by 70 publications

References 7 publications

A New Deep Learning Algorithm for SAR Scene Classification Based on Spatial Statistical Modeling and Features Re-Calibration

A New Deep Learning Algorithm for SAR Scene Classification Based on Spatial Statistical Modeling and Features Re-Calibration

Pivot Machine Translation Using Chinese as Pivot Language

From bilingual to multilingual neural‐based machine translation by incremental training

Contact Info

Product

Resources

About