Effect of Data Reduction on Sequence-to-sequence Neural TTS

Latorre, Javier; Lachowicz, Jakub; Lorenzo-Trueba, Jaime; Merritt, Thomas; Drugman, Thomas; Ronanki, Srikanth; Klimkov, Viacheslav

doi:10.1109/icassp.2019.8682168

Cited by 42 publications

(32 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each Tacotron was trained for 350k training steps. As [24] noted, neural S2S TTS models occasionally fail to stabilise, as differing random seeds influence the ability to learn effective alignments between text and speech. Some Tacotrons were unstable after training, therefore we trained each system with 3 different seeds and, after informal listening, selected the best performing for the MUSHRA.…”

Section: Tts Systemsmentioning

confidence: 99%

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

Taylor¹,

Richmond

2020

Interspeech 2020

View full text Add to dashboard Cite

Neural sequence-to-sequence (S2S) modelling encodes a single, unified representation for each input sequence. When used for text-to-speech synthesis (TTS), such representations must embed ambiguities between English spelling and pronunciation. For example, in pothole and there the character sequence th sounds different. This can be problematic when predicting pronunciation directly from letters. We posit pronunciation becomes easier to predict when letters are grouped into subword units like morphemes (e.g. a boundary lies between t and h in pothole but not there). Moreover, morphological boundaries can reduce the total number of, and increase the counts of, seen unit subsequences. Accordingly, we test here the effect of augmenting input sequences of letters with morphological boundaries. We find morphological boundaries substantially lower the Word and Phone Error Rates (WER and PER) for a Bi-LSTM performing G2P on one hand, and also increase the naturalness scores of Tacotrons performing TTS in a MUSHRA listening test on the other. The improvements to TTS quality are such that grapheme input augmented with morphological boundaries outperforms phone input without boundaries. Since morphological segmentation may be predicted with high accuracy, we highlight this simple pre-processing step has important potential for S2S modelling in TTS.

show abstract

Section: Tts Systemsmentioning

confidence: 99%

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

Taylor¹,

Richmond

2020

Interspeech 2020

View full text Add to dashboard Cite

show abstract

“…Voice banking is a simple idea of collecting a patient's speech samples before their speech becomes unintelligible and using it to build a personalized Text-To-Speech (TTS) voice. It requires about 1800 utterances for a basic unitselection TTS technology [14] and more than 5K utterances for building a Neural TTS voice [15]. Voice adaptation requires as little as 7 minutes of recordings.…”

Section: Speech Reconstructionmentioning

confidence: 99%

Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech

et al. 2019

Self Cite

View full text Add to dashboard Cite

We present a novel deep learning model for the detection and reconstruction of dysarthric speech. We train the model with a multi-task learning technique to jointly solve dysarthria detection and speech reconstruction tasks. The model key feature is a low-dimensional latent space that is meant to encode the properties of dysarthric speech. It is commonly believed that neural networks are black boxes that solve problems but do not provide interpretable outputs. On the contrary, we show that this latent space successfully encodes interpretable characteristics of dysarthria, is effective at detecting dysarthria, and that manipulation of the latent space allows the model to reconstruct healthy speech from dysarthric speech. This work can help patients and speech pathologists to improve their understanding of the condition, lead to more accurate diagnoses and aid in reconstructing healthy speech for afflicted patients.

show abstract

“…It uses a per-language encoder to process input sequences in terms of phonemes. [10] uses a phoneme-based representation with vowels using three different symbols depending on their level of stress.…”

Section: Introductionmentioning

confidence: 99%

Building Multilingual End-to-End Speech Synthesisers for Indian Languages

Prakash¹,

Thomas²,

Umesh³

et al. 2019

10th ISCA Workshop on Speech Synthesis (SSW 10)

View full text Add to dashboard Cite

Building text-to-speech (TTS) synthesisers is a difficult task, especially for low resource languages. Language-specific modules need to be developed for system building. End-to-end speech synthesis has become a popular paradigm as a TTS can be trained using only pairs. However, end-to-end speech synthesis is not scalable in a multilanguage scenario, as the vocabulary increases with the number of different scripts. In this paper, TTSes are trained for Indian languages using two text representations-character-based and phone-based. For the character-based approach, a multi-language character map (MLCM) is proposed to easily train Indic speech synthesisers. The phone-based approach uses the common label set (CLS) representation for Indian languages. Both approaches leverage the similarities that exist among the languages. The advantage is a compact representation across multiple languages. Experiments are conducted by building TTSes using monolingual data and by pooling data across two languages. The ability to synthesise code-mixed text using the phone-based approach is also assessed. Subjective evaluations indicate that reasonably good quality Indic TTSes can be developed using both approaches. This emphasises the need to incorporate multilingual text processing in the end-to-end framework.

show abstract

Effect of Data Reduction on Sequence-to-sequence Neural TTS

Cited by 42 publications

References 16 publications

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech

Building Multilingual End-to-End Speech Synthesisers for Indian Languages

Contact Info

Product

Resources

About