Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition

Cai, Xiong; Dai, Dongyang; Wu, Zhiyong; Li, Xiang; Li, Jingbei; Meng, Helen

doi:10.48550/arxiv.2010.13350

Cited by 2 publications

(2 citation statements)

References 22 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally, with a well-trained neural acoustic model [2,5,6,7] and a neural vocoder [8,9,10,11], or alternatively using fully end-to-end models [12,13,14] which directly construct wave signals from text input, it is able to synthesize high-quality neutral speech. Recently, much attention has been attracted to synthesizing expressive speech, such as stylized speech [15,16], emotional speech [17,18,19,20,21,22], and also singing voice [23,24].…”

Section: Introductionmentioning

confidence: 99%

Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis

Wang¹,

Wang²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper introduces Opencpop, a publicly available highquality Mandarin singing corpus designed for singing voice synthesis (SVS). The corpus consists of 100 popular Mandarin songs performed by a female professional singer. Audio files are recorded with studio quality at a sampling rate of 44,100 Hz and the corresponding lyrics and musical scores are provided. All singing recordings have been phonetically annotated with phoneme boundaries and syllable (note) boundaries. To demonstrate the reliability of the released data and to provide a baseline for future research, we built baseline deep neural network-based SVS models and evaluated them with both objective metrics and subjective mean opinion score (MOS) measure. Experimental results show that the best SVS model trained on our database achieves 3.70 MOS, indicating the reliability of the provided corpus. Opencpop is released to the open-source community WeNet 1 , and the corpus, as well as synthesized demos, can be found on the project homepage 2 .

show abstract

Section: Introductionmentioning

confidence: 99%

Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis

Wang¹,

Wang²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Wu's work [31] uses the weight of the attention layer in the GST as the probability of each emotion label to calculate the crossentropy loss and generate the emotion token. Some studies [32] use the speech emotion recognition module as an alternative approach to modeling the emotion feature from existing audio. Several other studies [33,34] use VAE or DurIAN to model the emotional information and have achieved some progress.…”

Section: Introductionmentioning

confidence: 99%

EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model

Cui

Ren

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, there has been an increasing interest in neural speech synthesis. While the deep neural network achieves the stateof-the-art result in text-to-speech (TTS) tasks, how to generate a more emotional and more expressive speech is becoming a new challenge to researchers due to the scarcity of high-quality emotion speech dataset and the lack of advanced emotional TTS model. In this paper, we first briefly introduce and publicly release a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation. After that, we propose a simple but efficient architecture for emotional speech synthesis called EMSpeech. Unlike those models which need additional reference audio as input, our model could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding. In the experiment phase, we first validate the effectiveness of our dataset by an emotion classification task. Then we train our model on the proposed dataset and conduct a series of subjective evaluations. Finally, by showing a comparable performance in the emotional speech synthesis task, we successfully demonstrate the ability of the proposed model.

show abstract

Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition

Cited by 2 publications

References 22 publications

Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis

Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis

EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model

Contact Info

Product

Resources

About