Multilingual Neural Machine Translation for Low Resource Languages

Lakew, Surafel M.; Federico, Marcello; Negri, Matteo; Turchi, Marco

doi:10.4000/books.aaccademia.2419

Cited by 13 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These models use the vocabulary (word segmentation rules) of the init model, avoiding the proposed dynamic vocabulary. This fine-tuning approach is prevalent in continued model trainings, for adapting NMT models [37,38] or improving zero-shot and lowresource translation tasks [39,40,41]. For the alternative baseline where we fine-tune init with its static-vocabulary, we observed that results were mostly analogous to Bi-NMT models.…”

Section: Baseline Modelsmentioning

confidence: 84%

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

Lakew¹,

Erofeeva²,

Negri³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a method to transfer knowledge across neural machine translation (NMT) models by means of a shared dynamic vocabulary. Our approach allows to extend an initial model for a given language pair to cover new languages by adapting its vocabulary as long as new data become available (i.e., introducing new vocabulary items if they are not included in the initial model). The parameter transfer mechanism is evaluated in two scenarios: i) to adapt a trained single language NMT system to work with a new language pair and ii) to continuously add new language pairs to grow to a multilingual NMT system. In both the scenarios our goal is to improve the translation performance, while minimizing the training convergence time. Preliminary experiments spanning five languages with different training data sizes (i.e., 5k and 50k parallel sentences) show a significant performance gain ranging from +3.85 up to +13.63 BLEU in different language directions. Moreover, when compared with training an NMT model from scratch, our transfer-learning approach allows us to reach higher performance after training up to 4% of the total training steps.

show abstract

Section: Baseline Modelsmentioning

confidence: 84%

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

Lakew¹,

Erofeeva²,

Negri³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The motivation behind the proposed bottleneck is the success of the Transformer neural network [45] for speech processing. A Transformer neural network reduces the dimensionality and maintains the important features of speech signals.…”

Section: A Multi-head Self Attention Transformer With Time-frequency ...mentioning

confidence: 99%

Multi-Attention Bottleneck for Gated Convolutional Encoder-Decoder-Based Speech Enhancement

Saleem,

Gunawan,

Shafi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Convolutional encoder-decoder (CED) has emerged as a powerful architecture, particularly in speech enhancement (SE), which aims to improve the intelligibility and quality and intelligibility of noise-contaminated speech. This architecture leverages the strength of the convolutional neural networks (CNNs) in capturing high-level features. Usually, the CED architectures use the gated recurrent unit (GRU) or long-short-term memory (LSTM) as a bottleneck to capture temporal dependencies, enabling a SE model to effectively learn the dynamics and long-term temporal dependencies in the speech signal. However, Transformers neural networks with self-attention effectively capture long-term temporal dependencies. This study proposes a multi-attention bottleneck (MAB) comprised of a self-attention Transformer powered by a time-frequency attention (TFA) module followed by a channel attention module (CAM) to focus on the important features. The proposed bottleneck (MAB) is integrated into a CED architecture and named MAB-CED. The MAB-CED uses an encoder-decoder structure including a shared encoder and two decoders, where one decoder is dedicated to spectral masking and the other is used for spectral mapping. Convolutional Gated Linear Units (ConvGLU) and Deconvolutional Gated Linear Units (DeconvGLU) are used to construct the encoder-decoder framework. The outputs of two decoders are coupled by applying coherent averaging to synthesize the enhanced speech signal. The proposed speech enhancement is examined using two databases, VoiceBank+DEMAND and LibriSpeech. The results show that the proposed speech enhancement outperforms the benchmarks in terms of intelligibility and quality at various input SNRs. This indicates the performance of the proposed MAB-CED at improving the average PESQ by 0.55 (22.85%) with VoiceBank+DEMAND and by 0.58 (23.79%) with LibriSpeech. The average STOI is improved by 9.63% (VoiceBank+DEMAND) and 9.78% ( LibriSpeech) over the noisy mixtures.

show abstract

“…2012), or to aid geolocation prediction systems (Han, Cook, and Baldwin 2012). MT can address diatopic variation when translating between pairs of closely related languages (Nakov and Tiedemann 2012; Lakew, Cettolo, and Federico 2018), language varieties (Costa-jussà et al . 2018), and dialects (Zbib et al .…”

Section: Applicationsmentioning

confidence: 99%

“…2017) of neural MT, which yielded improvements over RNN-based seq2seq, for example, for Romanian–Italian and Dutch–German in bilingual, multilingual and zero-shot setups (Lakew et al . 2018).…”

Section: Applicationsmentioning

confidence: 99%

Natural language processing for similar languages, varieties, and dialects: A survey

Zampieri

Nakov²,

Scherrer

2020

Nat. Lang. Eng.

View full text Add to dashboard Cite

There has been a lot of recent interest in the natural language processing (NLP) community in the computational processing of language varieties and dialects, with the aim to improve the performance of applications such as machine translation, speech recognition, and dialogue systems. Here, we attempt to survey this growing field of research, with focus on computational methods for processing similar languages, varieties, and dialects. In particular, we discuss the most important challenges when dealing with diatopic language variation, and we present some of the available datasets, the process of data collection, and the most common data collection strategies used to compile datasets for similar languages, varieties, and dialects. We further present a number of studies on computational methods developed and/or adapted for preprocessing, normalization, part-of-speech tagging, and parsing similar languages, language varieties, and dialects. Finally, we discuss relevant applications such as language and dialect identification and machine translation for closely related languages, language varieties, and dialects.

show abstract

Multilingual Neural Machine Translation for Low Resource Languages

Cited by 13 publications

References 13 publications

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

Multi-Attention Bottleneck for Gated Convolutional Encoder-Decoder-Based Speech Enhancement

Natural language processing for similar languages, varieties, and dialects: A survey

Contact Info

Product

Resources

About