Continuous Estimation of Emotions in Speech by Dynamic Cooperative Speaker Models

Abstract

Abstract-Automatic emotion recognition from speech has been recently focused on the prediction of time-continuous dimensions (e.g., arousal and valence) of spontaneous and realistic expressions of emotion, as found in real-life interactions. However, the automatic prediction of such emotions poses several challenges, such as the subjectivity found in the definition of a gold standard from a pool of raters and the issue of data scarcity in training models. In this work, we introduce a novel emotion recognition system, based on ensemble of single-speaker-regression-models (SSRMs). The estimation of emotion is provided by combining a subset of the initial pool of SSRMs selecting those that are most concordance among them. The proposed approach allows the addition or removal of speakers from the ensemble without the necessity to re-build the entire machine learning system. The simplicity of this aggregation strategy, coupled with the flexibility assured by the modular architecture, and the promising results obtained on the RECOLA database highlight the potential implications of the proposed method in a real-life scenario and in particular in WEB-based applications.

A closed-form solution to the graph total variation problem for continuous emotion profiling in noisy environment

“…ii) compared with the use of SYNC or GTVR alone, the combination of SYNC and GTVR (SGP) can improve the system's ability to predict emotions, which illustrates that it is important to reduce the annotation drift and environmental noise before predicting emotions; iii) the p-values of a paired t-test between the CCC values obtained on those three approaches show that the combined approach outperforms the performance reached by each single component; iv) in most of the cases, we obtain p < 0.001 between pairwise comparative methods for each dimension, verifying the effectiveness of the GTVR method. Moreover, the results illustrate that the proposed GTVR method is more than a mere denoising approach, but rather a signal approximator able to extract the relevant information from the underlying signal; v) consistently with the literature Mencattini et al (2017), the results show that arousal is better identified than valence. As an example, Fig.…”

“…With the development of artificial intelligence, humans have demanded more and more from affective computing, which facilitates the development of an increasing number of automatic speech emotion recognition (SER) applications and more relevantly dimensional emotion prediction from timecontinuous labels Gunes and Schuller (2013); Mencattini et al (2017); Martinelli et al (2016); Mariooryad and Busso (2015). Automatic emotion recognition (AER) technology from speech has matured well enough to be applied in some real-life scenarios Vignolo et al (2016), such as call centers Chen et al (2012), disease auxiliary diagnosis Schuller et al (2015), remote education and safe driving.…”

An array of physical sensors and an adaptive regression strategy for emotion recognition in a noisy scenario

“…(3) maximizes the mutual information between each feature 268 and the expected output, and it is also called Relevance Term. Further details can be found in [1,40]. 269…”

“…Several studies show that quantitative measurements of human expression can be used to estimate 48 psychological and physical conditions in humans [1,2,3]. Artificial empathic systems are expected to be of 49 benefit in many diverse domains such a precision medicine, personalized care and therapy, customer 50 satisfaction studies, or web profiling, to mention a few [4][5][6].…”

“…techniques then allow to embed in a unique sensorial system all the information related to the subjective 115 externalization of emotions using cooperative strategies able to combine descriptors with different range 116 and relevance levels [1,20]. 117…”