Emotional Voice Conversion Using a Hybrid Framework With Speaker-Adaptive DNN and Particle-Swarm-Optimized Neural Network

Abstract

We propose a hybrid network-based learning framework for speaker-adaptive vocal emotion conversion, tested on three different datasets (languages), namely, EmoDB (German), IITKGP (Telugu), and SAVEE (English). The optimized learning model introduced is unique because of its ability to synthesize emotional speech with an acceptable perceptive quality while preserving speaker characteristics. The multilingual model is extremely beneficial in scenarios wherein emotional training data from a specific target speaker are sparsely available. The proposed model uses speaker-normalized mel-generalized cepstral coefficients for spectral training with data adaptation using the seed data from the target speaker. The fundamental frequency (F0) is transformed using a wavelet synchrosqueezed transform prior to mapping to obtain a sharpened time-frequency representation. Moreover, a feedforward artificial neural network, together with particle swarm optimization, was used for F0 training. Additionally, static-intensity modification was also performed for each test utterance. Using the framework, we were able to capture the spectral and pitch contour variabilities of emotional expression better than with other state-of-the-art methods used in this study. Considering the overall performance scores across datasets, an average melcepstral distortion (MCD) of 4.98 and root mean square error (RMSE-F0) of 10.67 were obtained in objective evaluations, and an average comparative mean opinion score (CMOS) of 3.57 and speaker similarity score of 3.70 were obtained for the proposed framework. Particularly, the best MCD of 4.09 (EmoDB-happiness) and RMSE-F0 of 9.00 (EmoDB-anger) were obtained, along with the maximum CMOS of 3.7 and speaker similarity of 4.6, thereby highlighting the effectiveness of the hybrid network model.

Dementia Speech Dataset Creation and Analysis in Indic Languages—A Pilot Study

“…The results section highlights the important feature analysis conducted as part of the dataset creation and validation Subjective Mean Opinion Score Participants evaluate random speech samples Ranges from 1-5, (MOS) [77] from dataset for similarity 1-No similarity, 5-Exactly similar [78], [79] pipeline. The audio signals from English and Indic languages are subjected to dynamic time warping before the feature extraction process.…”

A New Approach for Speech Emotion Recognition Using Single Layered Convolutional Neural Network

“…Speaker recognition under noisy conditions using pipelined manners [19], is subject to computer vision techniques using CNN for confirming the speaker using facial recognition on the videos. Speech Emotion Recognition (SER) is approached from various perspectives, incorporating techniques such as gated recurrent unit (GRU) and multi-head attention [37][38][39]. These methods have been applied to the IEMOCAP and Emo-DB corpora, resulting in improved performance.…”

Research on Discourse Transfer Analysis Based on Deep Learning of Cross-language Transfer

“…In order to effectively describe the acoustic pattern information of speech in the time-frequency domain, model the long-term correlation of signals, and improve the naturalness of translated speech, a convolutional recurrent neural network with continuous wavelet transform is proposed in this paper. is CRNN model combined with the advantages of neural network, signal processing theory, and depth can use signal processing methods to obtain more suitable for the acoustic characteristics of the task and to make full use of the depth of the neural network nonlinear description ability to the words the local characteristics of spectrum and long correlation model, so as to achieve better performance of discourse transformation [18].…”