Uniform Multilingual Multi-Speaker Acoustic Model for Statistical Parametric Speech Synthesis of Low-Resourced Languages

Abstract

Acquiring data for text-to-speech (TTS) systems is expensive. This typically requires large amounts of training data, which is not available for low-resourced languages. Sometimes small amounts of data can be collected, while often no data may be available at all. This paper presents an acoustic modeling approach utilizing long short-term memory (LSTM) recurrent neural networks (RNN) aimed at partially addressing the language data scarcity problem. Unlike speaker-adaptation systems that aim to preserve speaker similarity across languages, the salient feature of the proposed approach is that, once constructed, the resulting system does not need retraining to cope with the previously unseen languages. This is due to language and speaker-agnostic model topology and universal linguistic feature set. Experiments on twelve languages show that the system is able to produce intelligible and sometimes natural output when a language is unseen. We also show that, when small amounts of training data are available, pooling the data sometimes improves the overall intelligibility and naturalness. Finally, we show that sometimes having a multilingual system with no prior exposure to the language is better than building single-speaker system from small amounts of data for that language. Index Terms: speech synthesis, low-resourced languages, long short-term memory, recurrent neural networks 2.1. Linguistic features Typically, the input to the acoustic model in statistical parametric speech synthesis consists of many diverse types of linguistic features. The features may include positional information (po

End-to-End Text-to-Speech for Low-Resource Languages by Cross-Lingual Transfer Learning

“…Previous research on multi-lingual multi-speaker (MLMS) statistical parametric speech synthesis (SPSS) has discussed using high-resource languages to help construct TTS systems for low-resource languages. Some research shows that the model trained on multiple languages can benefit from cross-lingual information and aid the adaptation to new languages using only a small amount of data [13,14]. In their methods, linguistic inputs of each language are converted internally into language-independent representations.…”

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

“…These resulting values (n = 880) were used for analysis. [6], [7], [8], [9], [10], [11], [12] Hidden Markov Model synthesis (HMM) 7 [12], [13], [14], [15], [16], [17], [18] Neural network (non-S2S) synthesis (DNN) 9 [19], [20], [21], [22], [23], [24], [25], [26], [27] Sequence-to-sequence synthesis (S2S)…”

“…Intelligibility [23] WER (Word Error Rate) Intelligibility [20] MOS (Mean Opinion Score) Naturalness/ Quality [6], [8], [9], [14], [15], [16], [17], [18], [21], [23] A/B Preference (preference rate b/w test & control) Quality [5], [10], [11], [12], [13]…”

An Investigation of Convolution Attention Based Models for Multilingual Speech Synthesis of Indian Languages

“…Previous work in exploring factorized multilingual, multispeaker neural models has been proposed in [12] and [14]. The model in [12] which in turn builds on top of the model proposed in [13], where data is factorized across speakers by having a separate speaker partitioned layer.…”

“…Thus, it allows us to explore various tying and weight sharing schemes between similar languages in case of lesser data, such as tying Assamese, Nepali and Bengali, while also allowing the use of a single model across all languages. Similarly the model in [14] also builds on this same multi-lingual, multi-speaker (MLMS) model for low-resource languages. However, both of these models still rely on frame-based linguistic features.…”