Prediction of Emotions from Text using Sentiment Analysis for Expressive Speech Synthesis

Abstract

The generation of expressive speech is a great challenge for text-to-speech synthesis in audiobooks. One of the most important factors is the variation in speech emotion or voice style. In this work, we developed a method to predict the emotion from a sentence so that we can convey it through the synthetic voice. It consists of combining a standard emotion-lexicon based technique with the polarity-scores (positive/negative polarity) provided by a less fine-grained sentiment analysis tool, in order to get more accurate emotion-labels. The primary goal of this emotion prediction tool was to select the type of voice (one of the emotions or neutral) given the input sentence to a stateof-the-art HMM-based Text-to-Speech (TTS) system. In addition, we also combined the emotion prediction from text with a speech clustering method to select the utterances with emotion during the process of building the emotional corpus for the speech synthesizer. Speech clustering is a popular approach to divide the speech data into subsets associated with different voice styles. The challenge here is to determine the clusters that map out the basic emotions from an audiobook corpus that contains high variety of speaking styles, in a way that minimizes the need for human annotation. The evaluation of emotion classification from text showed that, in general, our system can obtain accuracy results close to that of human annotators. Results also indicate that this technique is useful in the selection of utterances with emotion for building expressive synthetic voices.

An Active Data Representation of Videos for Automatic Scoring of Oral Presentation Delivery Skills and Feedback Generation

“…There is no prior knowledge available on the number of clusters in students' presentations. However, a study conducted on audio books data suggests that there are 50 different clusters (types of spoken expressions, such as spoken emotion or voice style) in those data (Vanmassenhove et al, 2016). Therefore, we used different number of clusters (m = 5, 10, 15, ..., 100) for generating the audio data representation.…”

Cross-corpus Feature Learning between Spontaneous Monologue and Dialogue for Automatic Classification of Alzheimer’s Dementia Speech

“…SOM is an attractive clustering method in this context as it addresses both topology and distribution, and requires no assumptions regarding the input vectors. Furthermore, it has been previously used for speech segment clustering based on voice styles with good results [30], [31]. Here m represents the number of SOM clusters that correspond to the FEM.…”

Unsupervised Learning for Expressive Speech Synthesis

“…For instance, Trilla and Alias (2013) already used sentiment analysis on sentence level for an expressive TTS. Vanmassenhove et al (2016) also used sentiment combined with emotion labels for an HMM-based system. Sudhakar and Bensraj (2014) implemented a TTS in Matlab which used sentiment information trained with fuzzy neural networks evaluated in a news domain.…”