Interactive Text-to-Speech System via Joint Style Analysis

Gao, Yang; Zheng, Wenlong; Yang, Zhaojun; Köhler, Thilo; Fuegen, Christian; He, Qianhua

doi:10.48550/arxiv.2002.06758

2020

DOI: 10.48550/arxiv.2002.06758

|View full text |Cite

Preprint

Interactive Text-to-Speech System via Joint Style Analysis

Yang Gao¹,

Wenlong Zheng²,

Zhaojun Yang³

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2020

2021

Publication Types

Select...

Other5

Relationship

Self Cite0

Independent5

Authors

Journals

Cited by 5 publications

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Expressive TTS Training with Frame and Style Reconstruction Loss

Liu

Şişman

Gao

et al. 2020

Preprint

View full text Add to dashboard Cite

We propose a novel training strategy for Tacotronbased text-to-speech (TTS) system to improve the expressiveness of speech. One of the key challenges in prosody modeling is the lack of reference that makes explicit modeling difficult. The proposed technique doesn't require prosody annotations from training data. It doesn't attempt to model prosody explicitly either, but rather encodes the association between input text and its prosody styles using a Tacotron-based TTS framework. Our proposed idea marks a departure from the style token paradigm where prosody is explicitly modeled by a bank of prosody embeddings. The proposed training strategy adopts a combination of two objective functions: 1) frame level reconstruction loss, that is calculated between the synthesized and target spectral features; 2) utterance level style reconstruction loss, that is calculated between the deep style features of synthesized and target speech. The proposed style reconstruction loss is formulated as a perceptual loss to ensure that utterance level speech style is taken into consideration during training. Experiments show that the proposed training strategy achieves remarkable performance and outperforms a state-of-the-art baseline in both naturalness and expressiveness. To our best knowledge, this is the first study to incorporate utterance level perceptual quality as a loss function into Tacotron training for improved expressiveness.

show abstract

Expressive TTS Training with Frame and Style Reconstruction Loss

Liu

Şişman

Gao

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition

Cai

Dai

et al. 2020

Preprint

View full text Add to dashboard Cite

Neural text-to-speech (TTS) approaches generally require a huge number of high quality speech data, which makes it difficult to obtain such a dataset with extra emotion labels. In this paper, we propose a novel approach for emotional TTS synthesis on a TTS dataset without emotion labels. Specifically, our proposed method consists of a cross-domain speech emotion recognition (SER) model and an emotional TTS model. Firstly, we train the cross-domain SER model on both SER and TTS datasets. Then, we use emotion labels on the TTS dataset predicted by the trained SER model to build an auxiliary SER task and jointly train it with the TTS model. Experimental results show that our proposed method can generate speech with the specified emotional expressiveness and nearly no hindering on the speech quality.

show abstract

Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset

Zhou

Şişman

et al. 2020

Preprint

View full text Add to dashboard Cite

Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity.Prior studies show that it is possible to disentangle emotional prosody using an encoder-decoder network conditioned on discrete representation, such as one-hot emotion labels. Such networks learn to remember a fixed set of emotional styles. In this paper, we propose a novel framework based on variational auto-encoding Wasserstein generative adversarial network (VAW-GAN), which makes use of a pre-trained speech emotion recognition (SER) model to transfer emotional style during training and at run-time inference. In this way, the network is able to transfer both seen and unseen emotional style to a new utterance. We show that the proposed framework achieves remarkable performance by consistently outperforming the baseline framework. This paper also marks the release of an emotional speech dataset (ESD) for voice conversion, which has multiple speakers and languages.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Interactive Text-to-Speech System via Joint Style Analysis

Cited by 5 publications

References 18 publications

Expressive TTS Training with Frame and Style Reconstruction Loss

Expressive TTS Training with Frame and Style Reconstruction Loss

Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition

Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset

Contact Info

Product

Resources

About