Dimensional Affective Speech Synthesis Based on Voice Conversion

Abstract

Affective speech synthesis can promote more natural human–computer interaction. Previous studies in the field of speech synthesis have used feature conversion to achieve natural affective speech. However, they focused on the adjustment of prosodic features and typically used a discrete emotion model; few studies on affective speech synthesis reflect the dimensional emotions expressed in daily life. To address these issues, we introduce a 2-dimensional valence–arousal emotion model into a speech synthesis system and take inspiration from voice conversion to convert prosodic and spectral acoustic features to achieve dimensional emotional speech expression. First, the acoustic features corresponding to the input text are predicted by the front end of the speech synthesis system, or the acoustic features of the input speech are extracted by World, a vocoder-based speech synthesis tool that generates prosodic and spectral features simultaneously. Then, the acoustic features of different dimensions of affective speech are analyzed and the fundamental frequency parameters and spectral envelope parameters of the source speech are converted based on the average ratio of the acoustic features of the input speech and the affective dimensions of the target. Finally, the World vocoder is used to output the converted emotion feature parameters into audio waveforms, and emotional speech synthesis with different dimensional values is realized in the 2-dimensional valence–arousal space. Objective and subjective evaluation results show that the dimensional affective speech synthesized using this method can be perceived well, especially in the arousal dimension.