Expressive Speech Synthesis Using Sentiment Embeddings

Abstract

In this paper we present a DNN based speech synthesis system trained on an audiobook including sentiment features predicted by the Stanford sentiment parser. The baseline system uses DNN to predict acoustic parameters based on conventional linguistic features, as they have been used in statistical parametric speech synthesis. The predicted parameters are transformed into speech using a conventional high-quality vocoder. In this paper, the conventional linguistic features are enriched using sentiment features. Different sentiment representations have been considered, combining sentiment probabilities with hierarchical distance and context. After preliminary analysis a listening experiment is conducted, where participants evaluate the different systems. The results show the usefulness of the proposed features and reveal differences between expert and non-expert TTS user.

Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS

“…The study on expressive speech synthesis is focused on prosody modeling [22]- [24], where speech prosody generally refers to intonation, stress, speaking rate, and phrase breaks. Prosodic phrasing [25]- [28] plays an important role in both affective and linguistic expressions.…”

Sentiment Analysis for Emotional Speech Synthesis in a News Dialogue System

“…On the other hand, in audiobooks and news reading systems, the emotions when the system speaks are often determined by the content of the text. In such applications, it is desirable that the emotional parameters used in the speech synthesis system be automatically estimated from the input text (Bellegarda, 2011;Jauk et al, 2018;Shaikh et al, 2009;Sudhakar and Bensraj, 2014;Trilla and Alías, 2013;Vanmassenhove et al, 2016).…”