An investigation to transplant emotional expressions in DNN-based TTS synthesis

Inoue, Katsuki; Hara, Sunao; Abe, Masanobu; Hojo, Nobukatsu; Ijima, Yusuke

doi:10.1109/apsipa.2017.8282231

Cited by 26 publications

(15 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Using input codes for representing different styles is also presented in [119, 120]. There have also been attempts at style transplantation, i.e., producing speech in the voice of speaker A in style X without having any sentence from speaker A in style X in the training data, in which case the network is forced to learn the style X from other speakers in the training database [121, 122].…”

Section: Progress In Speech Recognition and Synthesis As Well As mentioning

confidence: 99%

Speech Technology Progress Based on New Machine Learning Paradigm

Delić

Perić

Secujski

et al. 2019

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Speech technologies have been developed for decades as a typical signal processing area, while the last decade has brought a huge progress based on new machine learning paradigms. Owing not only to their intrinsic complexity but also to their relation with cognitive sciences, speech technologies are now viewed as a prime example of interdisciplinary knowledge area. This review article on speech signal analysis and processing, corresponding machine learning algorithms, and applied computational intelligence aims to give an insight into several fields, covering speech production and auditory perception, cognitive aspects of speech communication and language understanding, both speech recognition and text-to-speech synthesis in more details, and consequently the main directions in development of spoken dialogue systems. Additionally, the article discusses the concepts and recent advances in speech signal compression, coding, and transmission, including cognitive speech coding. To conclude, the main intention of this article is to highlight recent achievements and challenges based on new machine learning paradigms that, over the last decade, had an immense impact in the field of speech signal processing.

show abstract

Section: Progress In Speech Recognition and Synthesis As Well As mentioning

confidence: 99%

Speech Technology Progress Based on New Machine Learning Paradigm

Delić

Perić

Secujski

et al. 2019

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…For generalizing emotional expressions to new speakers, early SPSS researches [13,14] tried to represent emotion as an additive factors. [15] investigated several deep neural network (DNN) architectures for emotion transplantation. In this work they used 3 emotional and 21 neutral speakers.…”

Section: Multi-speaker Essmentioning

confidence: 99%

Multi-Speaker Emotional Speech Synthesis with Fine-Grained Prosody Modeling

Chunhui

Wen

Liu

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We present an end-to-end system for multi-speaker emotional speech synthesis. In particular, our system learns emotion classes from just two speakers then generalizes these classes to other speakers from whom no emotional data was seen. We address the problem by integrating disentangled, fine-grained prosody features with global, sentence-level emotion embedding. These fine-grained features learn to represent local prosodic variations disentangled from speaker, tone and global emotion label. Compared to systems that model emotions at sentence level only, our method achieves higher ratings in naturalness and expressiveness, while retaining comparable speaker similarity ratings.

show abstract

“…The first works in DNNs-based acoustic speech synthesis appeared in 2013 and used FeedForward DNNs to model the mapping between linguistic and acoustic features [12,29,30,31]. Later, other studies worked on adding expressiveness to the synthesized voice [32,20,33,34]. Regarding audiovisual speech, some works used DNNs to model emotion categories such as [35] and [16] who used FeedForward DNNs to synthesize expressive audiovisual speech.…”

Section: Related Workmentioning

confidence: 99%

Learning emotions latent representation with CVAE for text-driven expressive audiovisual speech synthesis

et al. 2021

View full text Add to dashboard Cite

Great improvement has been made in the field of expressive audiovisual Textto-Speech synthesis (EAVTTS) thanks to deep learning techniques. However, generating realistic speech is still an open issue and researchers in this area have been focusing lately on controlling the speech variability. In this paper, we use different neural architectures to synthesize emotional speech. We study the application of unsupervised learning techniques for emotional speech modeling as well as methods for restructuring emotions representation to make it continuous and more flexible. This manipulation of the emotional representation should allow us to generate new styles of speech by mixing emotions. We first present our expressive audiovisual corpus. We validate the emotional content of this corpus with three perceptual experiments using acoustic only, visual only and audiovisual stimuli. After that, we analyze the performance of a fully connected neural network in learning characteristics specific to different emotions for the phone duration aspect and the acoustic and visual modalities. We also study the contribution of a joint and separate training of the acoustic and visual modalities in the quality of the generated synthetic speech. In the second part of this paper, we use a conditional variational auto-encoder (CVAE) architecture to learn a latent representation of emotions. We applied this method in an unsupervised manner to generate features of expressive speech. We used a

show abstract

An investigation to transplant emotional expressions in DNN-based TTS synthesis

Cited by 26 publications

References 12 publications

Speech Technology Progress Based on New Machine Learning Paradigm

Speech Technology Progress Based on New Machine Learning Paradigm

Multi-Speaker Emotional Speech Synthesis with Fine-Grained Prosody Modeling

Learning emotions latent representation with CVAE for text-driven expressive audiovisual speech synthesis

Contact Info

Product

Resources

About