A DNN-based emotional speech synthesis by speaker adaptation

Yang, Hongwu; Zhang, Weizhao; Zhi, Pengpeng

doi:10.23919/apsipa.2018.8659599

Cited by 6 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RMSE is an established metric used [33], [34], [37], [49], [50], [52], [57] to evaluate the proximity of predicted values by a mapping algorithm to those of the target, given as…”

Section: ) Root Mean Square Errormentioning

confidence: 99%

“…Most of these strategies were based on a hidden Markov model using a constrained structural maximum a posteriori linear regression (CSMAPLR) adaptation [51] or training a speaker-dependent model by adapting from speakerindependent average models [52]. In addition, the method in [52] used a segment of duration as short as 5 min of the emotional data from the target speaker and could produce an appreciable perception of the synthesized emotion, however compromising on the speech quality. CSMAPLR considers the linguistic information from the regression tree, thereby distinguishing it from other adaptation approaches.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Emotional Voice Conversion Using a Hybrid Framework With Speaker-Adaptive DNN and Particle-Swarm-Optimized Neural Network

et al. 2020

View full text Add to dashboard Cite

We propose a hybrid network-based learning framework for speaker-adaptive vocal emotion conversion, tested on three different datasets (languages), namely, EmoDB (German), IITKGP (Telugu), and SAVEE (English). The optimized learning model introduced is unique because of its ability to synthesize emotional speech with an acceptable perceptive quality while preserving speaker characteristics. The multilingual model is extremely beneficial in scenarios wherein emotional training data from a specific target speaker are sparsely available. The proposed model uses speaker-normalized mel-generalized cepstral coefficients for spectral training with data adaptation using the seed data from the target speaker. The fundamental frequency (F0) is transformed using a wavelet synchrosqueezed transform prior to mapping to obtain a sharpened time-frequency representation. Moreover, a feedforward artificial neural network, together with particle swarm optimization, was used for F0 training. Additionally, static-intensity modification was also performed for each test utterance. Using the framework, we were able to capture the spectral and pitch contour variabilities of emotional expression better than with other state-of-the-art methods used in this study. Considering the overall performance scores across datasets, an average melcepstral distortion (MCD) of 4.98 and root mean square error (RMSE-F0) of 10.67 were obtained in objective evaluations, and an average comparative mean opinion score (CMOS) of 3.57 and speaker similarity score of 3.70 were obtained for the proposed framework. Particularly, the best MCD of 4.09 (EmoDB-happiness) and RMSE-F0 of 9.00 (EmoDB-anger) were obtained, along with the maximum CMOS of 3.7 and speaker similarity of 4.6, thereby highlighting the effectiveness of the hybrid network model.

show abstract

“…RMSE is an established metric used [33], [34], [37], [49], [50], [52], [57] to evaluate the proximity of predicted values by a mapping algorithm to those of the target, given as…”

Section: ) Root Mean Square Errormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Emotional Voice Conversion Using a Hybrid Framework With Speaker-Adaptive DNN and Particle-Swarm-Optimized Neural Network

et al. 2020

View full text Add to dashboard Cite

show abstract

“…By using a speaker adaptation method, Yang et. al [13] proposed a method that generated emotional speech from a small amount of emotional speech training data. In all aforementioned methods, the target speaker's emotional speech is necessary for training.…”

Section: Introductionmentioning

confidence: 99%

“…This works even if the target speaker's emotional speech is not included in training data. A key idea is to explicitly control the speaker factor and the emotional factor, motivated by the success in the multi-speaker model [6,7,8,9,10] and multi-emotional model [11,12,13]. Once the factors are trained, by independently controlling the factors, we can synthesize speech with any combination of a speaker and an emotion.…”

Section: Introductionmentioning

confidence: 99%

Model architectures to extrapolate emotional expressions in DNN-based text-to-speech

Inoue,

Hara,

Abe

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper proposes architectures that facilitate the extrapolation of emotional expressions in deep neural network (DNN)based text-to-speech (TTS). In this study, the meaning of "extrapolate emotional expressions" is to borrow emotional expressions from others, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power to construct DNN-based TTS with emotional expressions and some DNN-based TTS systems have demonstrated satisfactory performances in the expression of the diversity of human speech, it is necessary and troublesome to collect emotional speech uttered by target speakers. To solve this issue, we propose architectures to separately train the speaker feature and the emotional feature and to synthesize speech with any combined quality of speakers and emotions. The architectures are parallel model (PM), serial model (SM), auxiliary input model (AIM), and hybrid models (PM&AIM and SM&AIM). These models are trained through emotional speech uttered by few speakers and neutral speech uttered by many speakers. Objective evaluations demonstrate that the performances in the open-emotion test provide insufficient information. They make a comparison with those in the closed-emotion test, but each speaker has their own manner of expressing emotion. However, subjective evaluation results indicate that the proposed models could convey emotional information to some extent. Notably, the PM can correctly convey sad and joyful emotions at a rate of >60%.

show abstract

“…In recent years, research has not only improved the quality so that a synthesized voice is similar to a real one, but also has diversified expressions (Govind and Prasanna, 2013;Kaur and Singh, 2015;Skerry-Ryan et al, 2018). Emotional speech synthesis is a technique for diversifying the expression of speech synthesis (Qin et al, 2006;Schröder, 2001;Yang et al, 2018). It specifies both emotional parameters and the text input so that the speech reflects the designated emotion (Charfuelan and Steiner, 2013;Inoue et al, 2017;Iwata and Kobayashi, 2011;Nose and Kobayashi, 2013).…”

Section: Introductionmentioning

confidence: 99%

Sentiment Analysis for Emotional Speech Synthesis in a News Dialogue System

Takatsu¹,

Ando²,

Matsuyama³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

As smart speakers and conversational robots become ubiquitous, the demand for expressive speech synthesis has increased. In this paper, to control the emotional parameters of the speech synthesis according to certain dialogue contents, we construct a news dataset with emotion labels ("positive," "negative," or "neutral") annotated for each sentence. We then propose a method to identify emotion labels using a model combining BERT and BiLSTM-CRF, and evaluate its effectiveness using the constructed dataset. The results showed that the classification model performance can be efficiently improved by preferentially annotating news articles with low confidence in the human-in-the-loop machine learning framework.

show abstract

A DNN-based emotional speech synthesis by speaker adaptation

Cited by 6 publications

References 15 publications

Emotional Voice Conversion Using a Hybrid Framework With Speaker-Adaptive DNN and Particle-Swarm-Optimized Neural Network

Emotional Voice Conversion Using a Hybrid Framework With Speaker-Adaptive DNN and Particle-Swarm-Optimized Neural Network

Model architectures to extrapolate emotional expressions in DNN-based text-to-speech

Sentiment Analysis for Emotional Speech Synthesis in a News Dialogue System

Contact Info

Product

Resources

About