Deep Learning for Mandarin-Tibetan Cross-Lingual Speech Synthesis

Zhang, Weizhao; Yang, Hongwu; Bu, Xiaolong; Wang, Lili

doi:10.1109/access.2019.2954342

Cited by 19 publications

(12 citation statements)

References 22 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These resulting values (n = 880) were used for analysis. [6], [7], [8], [9], [10], [11], [12] Hidden Markov Model synthesis (HMM) 7 [12], [13], [14], [15], [16], [17], [18] Neural network (non-S2S) synthesis (DNN) 9 [19], [20], [21], [22], [23], [24], [25], [26], [27] Sequence-to-sequence synthesis (S2S)…”

Section: Characteristics Of the Included Studiesmentioning

confidence: 99%

“…From the chosen studies, notable studies include [18] (hereafter Study A), the latest included DNN-based study, which used 5 different evaluation metrics for TTS in Tibetan (with Mandarin as the source language). For S2S-based studies, among the latest are [27] (Study B), which explored TTS for Indic LRLs in the Indo-Aryan and Dravidian families, and [25] (Study C), which investigated strategies for using Dutch and other European languages to aid a limited amount of English data.…”

Section: Notable Studiesmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

Do¹,

Coler²,

Dijkstra³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

We provide a systematic review of past studies that use multilingual data for text-to-speech (TTS) of low-resource languages (LRLs). We focus on the strategies used by these studies for incorporating multilingual data and how they affect output speech quality. To investigate the difference in output quality between corresponding monolingual and multilingual models, we propose a novel measure to compare this difference across the included studies and their various evaluation metrics. This measure, called the Multilingual Model Effect (MLME), is found to be affected by: acoustic model architecture, the difference ratio of target language data between corresponding multilingual and monolingual experiments, the balance ratio of target language data to total data, and the amount of target language data used. These findings can act as reference for data strategies in future experiments with multilingual TTS models for LRLs. Language family classification, despite being widely used, is not found to be an effective criterion for selecting source languages.

show abstract

Section: Characteristics Of the Included Studiesmentioning

confidence: 99%

Section: Notable Studiesmentioning

confidence: 99%

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

Do¹,

Coler²,

Dijkstra³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

“…5, a structure including two different recurrent networks with the same output is capable of both forward and backward training process [36]. Proposed topology has been widely applied in the speech recognition domain due to its' capability to efficiently recognize a word by using not only the previous words but also the whole sentence [37]. Motivated by the proposed principle, here, the necessary information is completely exploited by the explanatory variables This work is licensed under a Creative Commons Attribution 4.0 License.…”

Section: Blstm Networkmentioning

confidence: 99%

Hybrid Deep Learning-Based Model for Wind Speed Forecasting Based on DWPT and Bidirectional LSTM Network

2020

View full text Add to dashboard Cite

Accurate wind speed forecasting is essential for the reliability and security of the power system, and optimal operation and management of wind integrated smart grids. However, it is still a challenging task due to the highly uncertain and volatile nature of wind speed. Accordingly, in this work, a novel deep learning-based model integrating the discrete wavelet packet transform (DWPT) and bidirectional long short-term memory (BLSTM) is developed to precisely capture deep temporal features and learn the time-varying relationship of wind speed time series. In the proposed method, by applying the DWPT, both approximations and details parts are decomposed by passing through the filters to choose the frequency band related to the features of the original signal more adaptively. The BLSTM networks are incorporated to deal with the uncertainties more effectively as they have bidirectional memory capability (feedforward and feedback loops) to investigate both previous and future hidden layers data. To simultaneously improve the forecasting performance and decrease the learning complexity, the reconstructed state space of historical wind data is employed to reflect the evolution laws of wind speed. Two case studies using real-world wind speed datasets gathered from Flatirons campus (M2) of National Renewable Energy Laboratory (NREL) located in Colorado, USA and weather station of Edmonton, Canada are implemented to demonstrate the effectiveness and superiority of the proposed hybrid method compared to the shallow architectures and state-of-the-art deep learning models in the recent literature.INDEX TERMS Bidirectional LSTM, deep learning, time series, Takens' embedding theorem, wavelet packet decomposition, wind speed prediction.

show abstract

“…e experimental results show that synthesized Tibetan speech is better than the HMM-based Mandarin-Tibetan cross-lingual speech synthesis. e work [20] trains the acoustic models with DNN, hybrid long short-term memory (LSTM), and hybrid bidirectional long short-term memory (BLSTM) and implements a deep learning-based Mandarin-Tibetan crosslingual speech synthesis under a unique framework. Experiments demonstrated that the hybrid BLSTM-based cross-lingual speech synthesis framework was better than the Tibetan monolingual framework.…”

Section: Introductionmentioning

confidence: 99%

End‐to‐End Speech Synthesis for Tibetan Multidialect

Zhao

et al. 2021

Complexity

View full text Add to dashboard Cite

The research on Tibetan speech synthesis technology has been mainly focusing on single dialect, and thus there is a lack of research on Tibetan multidialect speech synthesis technology. This paper presents an end-to-end Tibetan multidialect speech synthesis model to realize a speech synthesis system which can be used to synthesize different Tibetan dialects. Firstly, Wylie transliteration scheme is used to convert the Tibetan text into the corresponding Latin letters, which effectively reduces the size of training corpus and the workload of front-end text processing. Secondly, a shared feature prediction network with a cyclic sequence-to-sequence structure is built, which maps the Latin transliteration vector of Tibetan character to Mel spectrograms and learns the relevant features of multidialect speech data. Thirdly, two dialect-specific WaveNet vocoders are combined with the feature prediction network, which synthesizes the Mel spectrum of Lhasa-Ü-Tsang and Amdo pastoral dialect into time-domain waveform, respectively. The model avoids using a large number of Tibetan dialect expertise for processing some time-consuming tasks, such as phonetic analysis and phonological annotation. Additionally, it can directly synthesize Lhasa-Ü-Tsang and Amdo pastoral speech on the existing text annotation. The experimental results show that the synthesized speech of Lhasa-Ü-Tsang and Amdo pastoral dialect based on our proposed method has better clarity and naturalness than the Tibetan monolingual model.

show abstract

Deep Learning for Mandarin-Tibetan Cross-Lingual Speech Synthesis

Cited by 19 publications

References 22 publications

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

Hybrid Deep Learning-Based Model for Wind Speed Forecasting Based on DWPT and Bidirectional LSTM Network

End‐to‐End Speech Synthesis for Tibetan Multidialect

Contact Info

Product

Resources

About