Nonparallel Emotional Speech Conversion Using VAE-GAN

Cao, Yue; Liu, Zhengchen; Chen, Minchuan; Ma, Jun; Wang, Shaojun; Xiao, Jing

doi:10.21437/interspeech.2020-1647

Cited by 14 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The generator learns to reconstruct the input features from the latent code z and the emotion identity, the discriminator learns to distinguish the reconstructed features with the real features. Autoencoder is easy to train owing to its simple architecture [73,50,49].…”

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

“…The generator is then conditioned on such deep emotional feature or one-hot emotion identity during the training. In this way, we learn the framelevel content-related representations from the speech in an unsupervised manner [50,60,48]. VAE-GAN [50] and VAW-GAN [25,60] are successful attempts.…”

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

“…In this way, we learn the framelevel content-related representations from the speech in an unsupervised manner [50,60,48]. VAE-GAN [50] and VAW-GAN [25,60] are successful attempts. Another idea is for an autoencoder to learn an emotion-invariant latent code and an emotion-related style code in latent space [49].…”

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

“…The recent studies on deep learning have seen remarkable performance, such as DNN [16,39,40], highway neural network [41], deep bi-directional long-short-term memory network (DBLSTM) [42], and sequence-to-sequence model [43,44]. Beyond parallel training data, new techniques have been proposed to learn the translation between emotional domains with CycleGAN [45,46] and StarGAN [47], to disentangle the emotional elements from speech with auto-encoders [48,49,50,51], and to leverage text-to-speech (TTS) [52,53] or automatic speech recognition (ASR) [54]. Such framework generally works well in speaker-dependent tasks.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Emotional Voice Conversion: Theory, Databases and ESD

Zhou¹,

Şişman²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we first provide a review of the state-of-the-art emotional voice conversion research, and the existing emotional speech databases. We then motivate the development of a novel emotional speech database (ESD) that addresses the increasing research need. With this paper, the ESD database 1 is now made available to the research community. The ESD database consists of 350 parallel utterances spoken by 10 native English and 10 native Chinese speakers and covers 5 emotion categories (neutral, happy, angry, sad and surprise). More than 29 hours of speech data were recorded in a controlled acoustic environment. The database is suitable for multi-speaker and cross-lingual emotional voice conversion studies. As case studies, we implement several state-of-the-art emotional voice conversion systems on the ESD database. This paper provides a reference study on ESD in conjunction with its release.

show abstract

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

Section: Disentanglement Between Emotional Prosody and Linguistic Con...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Emotional Voice Conversion: Theory, Databases and ESD

Zhou¹,

Şişman²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Lately, Ding et al [12] adopted vector quantized variational autoencoders (VQ-VAE) with the group latent embedding (GLE) for nonparallel data training. Moreover, to better learn the mapping function between non-parallel data distributions, cycle-consistent adversarial network (CycleGAN) [13,14] and variational autoencoder-generative adversarial network (VAE-GAN) [15] were introduced to EVC task. Furthermore, Moritani et al [16] employed starGAN to realize non-parallel spectral envelope transformation.…”

Section: Introductionmentioning

confidence: 99%

CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer

Liu²,

Ishi³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this study, we explore the transformer's ability to capture intra-relations among frames by augmenting the receptive field of models. Concretely, we propose a CycleGAN-based model with the transformer and investigate its ability in the emotional voice conversion task. In the training procedure, we adopt curriculum learning to gradually increase the frame length so that the model can see from the short segment till the entire speech. The proposed method was evaluated on the Japanese emotional speech dataset and compared to several baselines (ACVAE, CycleGAN) with objective and subjective evaluations. The results show that our proposed model is able to convert emotion with higher strength and quality.

show abstract

Non-parallel Voice Conversion Based on Perceptual Star Generative Adversarial Network

Qiu

Cao

et al. 2022

Circuits Syst Signal Process

View full text Add to dashboard Cite

Nonparallel Emotional Speech Conversion Using VAE-GAN

Cited by 14 publications

References 15 publications

Emotional Voice Conversion: Theory, Databases and ESD

Emotional Voice Conversion: Theory, Databases and ESD

CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer

Non-parallel Voice Conversion Based on Perceptual Star Generative Adversarial Network

Contact Info

Product

Resources

About