High Quality, Lightweight and Adaptable TTS Using LPCNet

Kons, Zvi; Shechtman, Slava; Sorin, Alexander; Rabinovitz, Carmel; Hoory, Ron

doi:10.21437/interspeech.2019-1705

Cited by 38 publications

(26 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the [TEXT] setting, unlike the other two settings described above, we assume that transcripts and semantic labels are available but corresponding audio is absent. To circumvent the lack of real speech, as in [8,9], we use a TTS system [31] to synthesize speech from the transcripts and then use the speech to perform SLU training. Given that synthesized speech can be quite different from the test audio, we 4).…”

Section: Slu Models In the [Text] Settingmentioning

confidence: 99%

RNN Transducer Models for Spoken Language Understanding

Thomas¹,

Kuo²,

Saon³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We present a comprehensive study on building and adapting RNN transducer (RNN-T) models for spoken language understanding (SLU). These end-to-end (E2E) models are constructed in three practical settings: a case where verbatim transcripts are available, a constrained case where the only available annotations are SLU labels and their values, and a more restrictive case where transcripts are available but not corresponding audio. We show how RNN-T SLU models can be developed starting from pre-trained automatic speech recognition (ASR) systems, followed by an SLU adaptation step. In settings where real audio data is not available, artificially synthesized speech is used to successfully adapt various SLU models. When evaluated on two SLU data sets, the ATIS corpus and a customer call center data set, the proposed models closely track the performance of other E2E models and achieve state-of-the-art results.

show abstract

Section: Slu Models In the [Text] Settingmentioning

confidence: 99%

RNN Transducer Models for Spoken Language Understanding

Thomas¹,

Kuo²,

Saon³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The TTS system architecture is similar to the single speaker system described in [25]. It is a modular system based on three neural-net models: one to infer prosody, one to infer acoustic features, and an LPCNet [26] vocoder.…”

Section: Tts Systemmentioning

confidence: 99%

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Huang

Kuo

Thomas

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is time consuming and expensive to collect. Initializing the S2I model with an ASR model trained on copious speech data can alleviate data sparsity. In this paper, we attempt to leverage NLU text resources. We implemented a CTC-based S2I system that matches the performance of a state-ofthe-art, traditional cascaded SLU system. We performed controlled experiments with varying amounts of speech and text training data. When only a tenth of the original data is available, intent classification accuracy degrades by 7.6% absolute. Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the textto-intent data is converted into speech-to-intent data using a multispeaker text-to-speech system. The proposed approaches recover 80% of performance lost due to using limited intent-labeled speech.

show abstract

“…Previous studies on few-shot TTS could be categorized into two general approaches. The first approach pre-tains multi-speaker TTS models on a large multi-speaker dataset and then fine-tunes the models on a small dataset of target speaker [7,8]. The second approach predicts a speaker embedding from speech to clone unseen speakers without fine-tuning.…”

Section: Introductionmentioning

confidence: 99%

Investigation of Fast and Efficient Methods for Multi-Speaker Modeling and Speaker Adaptation

Zheng

Lü

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we propose a novel method for fast and efficient fewshot TTS task, which is able to disentangle linguistic and speaker representations. Specifically, an adversarial training strategy is firstly employed to wipe out speaker information from the linguistic representations. Then the speaker representations are extracted from audio signals by a speaker encoder with a random sampling mechanism and a speaker classifier, aiming to extract speaker embedding features that are independent of content information (such as prosody and style etc). Meanwhile, for faster and efficient adaptation, we further introduce the prior alignment knowledge between the text and audio pairs and propose a multi-alignment guided attention to help the attention learning. The Experimental results show the proposed method not only could generate higher speech quality and speaker similarity with an average absolute improvement of 0.26 and 0.30 in MOS respectively, when adapting to new speakers with 20 utterances, but also converge much faster and efficient. Moreover, we can achieve a MOS of 4.45 for a premium voice, which outperforms a single speaker model of 4.23. 1

show abstract

High Quality, Lightweight and Adaptable TTS Using LPCNet

Cited by 38 publications

References 12 publications

RNN Transducer Models for Spoken Language Understanding

RNN Transducer Models for Spoken Language Understanding

Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems

Investigation of Fast and Efficient Methods for Multi-Speaker Modeling and Speaker Adaptation

Contact Info

Product

Resources

About