Learning cross-lingual information with multilingual BLSTM for speech synthesis of low-resource languages

Yu, Quanjie; Liu, Peng; Wu, Zhiyong; Ang, Shiyin K; Meng, Helen; Cai, Lianhong

doi:10.1109/icassp.2016.7472738

Cited by 30 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous research on multi-lingual multi-speaker (MLMS) statistical parametric speech synthesis (SPSS) has discussed using high-resource languages to help construct TTS systems for low-resource languages. Some research shows that the model trained on multiple languages can benefit from cross-lingual information and aid the adaptation to new languages using only a small amount of data [13,14]. In their methods, linguistic inputs of each language are converted internally into language-independent representations.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Text-to-Speech for Low-Resource Languages by Cross-Lingual Transfer Learning

Chen²,

Yeh

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

End-to-end text-to-speech (TTS) has shown great success on large quantities of paired text plus speech data. However, laborious data collection remains difficult for at least 95% of the languages over the world, which hinders the development of TTS in different languages. In this paper, we aim to build TTS systems for such low-resource (target) languages where only very limited paired data are available. We show such TTS can be effectively constructed by transferring knowledge from a high-resource (source) language. Since the model trained on source language cannot be directly applied to target language due to input space mismatch, we propose a method to learn a mapping between source and target linguistic symbols. Benefiting from this learned mapping, pronunciation information can be preserved throughout the transferring procedure. Preliminary experiments show that we only need around 15 minutes of paired data to obtain a relatively good TTS system. Furthermore, analytic studies demonstrated that the automatically discovered mapping correlate well with the phonetic expertise.

show abstract

Section: Introductionmentioning

confidence: 99%

End-to-End Text-to-Speech for Low-Resource Languages by Cross-Lingual Transfer Learning

Chen²,

Yeh

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

show abstract

“…The acoustic model is trained on multiple languages and may never observe the target language in its training data. This type of acoustic models, the multilingual multi-speaker (MLMS) models, were proposed in [12,13,14]. These approaches utilize a large input feature space consisting of "concatenated" language-dependent components, one for each language.…”

Section: Introductionmentioning

confidence: 99%

Uniform Multilingual Multi-Speaker Acoustic Model for Statistical Parametric Speech Synthesis of Low-Resourced Languages

Gutkin

2017

Interspeech 2017

View full text Add to dashboard Cite

Acquiring data for text-to-speech (TTS) systems is expensive. This typically requires large amounts of training data, which is not available for low-resourced languages. Sometimes small amounts of data can be collected, while often no data may be available at all. This paper presents an acoustic modeling approach utilizing long short-term memory (LSTM) recurrent neural networks (RNN) aimed at partially addressing the language data scarcity problem. Unlike speaker-adaptation systems that aim to preserve speaker similarity across languages, the salient feature of the proposed approach is that, once constructed, the resulting system does not need retraining to cope with the previously unseen languages. This is due to language and speaker-agnostic model topology and universal linguistic feature set. Experiments on twelve languages show that the system is able to produce intelligible and sometimes natural output when a language is unseen. We also show that, when small amounts of training data are available, pooling the data sometimes improves the overall intelligibility and naturalness. Finally, we show that sometimes having a multilingual system with no prior exposure to the language is better than building single-speaker system from small amounts of data for that language. Index Terms: speech synthesis, low-resourced languages, long short-term memory, recurrent neural networks 2.1. Linguistic features Typically, the input to the acoustic model in statistical parametric speech synthesis consists of many diverse types of linguistic features. The features may include positional information (po

show abstract

“…BLSTM-RNN is an extended architecture of bidirectional recurrent neural network (BRNN) [49]. It replaces units in the hidden layers of BRNN with LSTM memory blocks.…”

Section: Blstmmentioning

confidence: 99%

“…The notations of these equations are explained in [49] and φ(•) is the activation function which can be implemented by the LSTM block with equations in [49].…”

Section: Blstmmentioning

confidence: 99%

A Review of Deep Learning Based Speech Synthesis

Ning

et al. 2019

Applied Sciences

Self Cite

117

View full text Add to dashboard Cite

Speech synthesis, also known as text-to-speech (TTS), has attracted increasingly more attention. Recent advances on speech synthesis are overwhelmingly contributed by deep learning or even end-to-end techniques which have been utilized to enhance a wide range of application scenarios such as intelligent speech interaction, chatbot or conversational artificial intelligence (AI). For speech synthesis, deep learning based techniques can leverage a large scale of <text, speech> pairs to learn effective feature representations to bridge the gap between text and speech, thus better characterizing the properties of events. To better understand the research dynamics in the speech synthesis field, this paper firstly introduces the traditional speech synthesis methods and highlights the importance of the acoustic modeling from the composition of the statistical parametric speech synthesis (SPSS) system. It then gives an overview of the advances on deep learning based speech synthesis, including the end-to-end approaches which have achieved start-of-the-art performance in recent years. Finally, it discusses the problems of the deep learning methods for speech synthesis, and also points out some appealing research directions that can bring the speech synthesis research into a new frontier.

show abstract

Learning cross-lingual information with multilingual BLSTM for speech synthesis of low-resource languages

Cited by 30 publications

References 12 publications

End-to-End Text-to-Speech for Low-Resource Languages by Cross-Lingual Transfer Learning

End-to-End Text-to-Speech for Low-Resource Languages by Cross-Lingual Transfer Learning

Uniform Multilingual Multi-Speaker Acoustic Model for Statistical Parametric Speech Synthesis of Low-Resourced Languages

A Review of Deep Learning Based Speech Synthesis

Contact Info

Product

Resources

About