Cooperative Learning and its Application to Emotion Recognition from Speech

Zhang, Zixing; Coutinho, Eduardo; Deng, Jun; Schuller, Björn

doi:10.1109/taslp.2014.2375558

Cited by 75 publications

(74 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the PU-based DDAT system is slightly more important for valence prediction than it is for arousal prediction. This might be due to the fact that prediction of emotional valence is much more difficult than arousal for audio modality [2], [67], [68].…”

Section: Dynamic Tuning and Late Fusionmentioning

confidence: 99%

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Zhang

Han

Coutinho

et al. 2019

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (e. g., speech, face, and physiology). In this paper, motivated by the benefit of difficulty awareness in a human learning procedure, we propose a novel machine learning framework, namely, Dynamic Difficulty Awareness Training (DDAT), which sheds fresh light on the research -directly exploiting the difficulties in learning to boost the machine learning process. The DDAT framework consists of two stages: information retrieval and information exploitation. In the first stage, we make use of the reconstruction error of input features or the annotation uncertainty to estimate the difficulty of learning specific information. The obtained difficulty level is then used in tandem with original features to update the model input in a second learning stage with the expectation that the model can learn to focus on high difficulty regions of the learning process. We perform extensive experiments on a benchmark database (RECOLA) to evaluate the effectiveness of the proposed framework. The experimental results show that our approach outperforms related baselines as well as other well-established time-continuous emotion prediction systems, which suggests that dynamically integrating the difficulty information for neural networks can help enhance the learning process.

show abstract

Section: Dynamic Tuning and Late Fusionmentioning

confidence: 99%

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Zhang

Han

Coutinho

et al. 2019

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…The common approach in the literature is to use all the emotion variability found in the data as training material and tune the machine learning system in order to disregard the less relevant instances (e. g., by optimising the number of support vectors and the soft margin in Support Vector Regression (SVR)) for emotion prediction [19], [30], [31], [32]. Some recent work have proposed to use cooperative learning as a means to select the most informative instances from a set of unlabelled acoustic utterances [33]. But the core underlying idea of this approach is to reduce the cost of the human annotation task, e. g., by selecting instances which are predicted with a low confidence level, not to consider consensus as a way to optimise the predictability of a given SER system.…”

Section: Related Workmentioning

confidence: 99%

Continuous Estimation of Emotions in Speech by Dynamic Cooperative Speaker Models

Mencattini

Martinelli

Ringeval

et al. 2017

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Abstract-Automatic emotion recognition from speech has been recently focused on the prediction of time-continuous dimensions (e.g., arousal and valence) of spontaneous and realistic expressions of emotion, as found in real-life interactions. However, the automatic prediction of such emotions poses several challenges, such as the subjectivity found in the definition of a gold standard from a pool of raters and the issue of data scarcity in training models. In this work, we introduce a novel emotion recognition system, based on ensemble of single-speaker-regression-models (SSRMs). The estimation of emotion is provided by combining a subset of the initial pool of SSRMs selecting those that are most concordance among them. The proposed approach allows the addition or removal of speakers from the ensemble without the necessity to re-build the entire machine learning system. The simplicity of this aggregation strategy, coupled with the flexibility assured by the modular architecture, and the promising results obtained on the RECOLA database highlight the potential implications of the proposed method in a real-life scenario and in particular in WEB-based applications.

show abstract

“…Major attention is thus given to efficient procedures of data acquisition and annotation, such as by weakly supervised [28] or even unsupervised [34] learning approaches. As it is mostly the labels that are sparse rather than the data, crowd-sourcing and cooperative learning [35] are further popular recent assets to lead quickly to rich resources of annotated affective usage data. Similarly, transfer learning [7] can be used to adapt 'similar' data to the current IUI case.…”

Section: User Datamentioning

confidence: 99%

Modelling User Affect and Sentiment in Intelligent User Interfaces

Schuller

2015

Proceedings of the 20th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

The computer-based automatic analysis of human sentiment, and affect are broadly expected to play a major role that will likely make 'that difference' in future Intelligent User Interfaces, as they bear the promise to lend interactive systems emotional intelligence. Such comprise intelligent digital games, e. g., for empowerment and inclusion, tutoring systems, information systems or virtual companions, e. g., in the car to name but a few. This tutorial aims to give a good introduction into the related fields of user Sentiment Analysis and user Affect Modelling. Its intention is to show the general technology, and its current reliability, the ways for technical integration and efficient embedding of solutions in a user interface context, and the latest trends in this young and ever emerging field. Emphasis is laid on highlighting the range of toolkits available at this moment with the aim of empowering one to immediately craft own solutions. This description contains the general motivation,

show abstract

Cooperative Learning and its Application to Emotion Recognition from Speech

Cited by 75 publications

References 37 publications

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Continuous Estimation of Emotions in Speech by Dynamic Cooperative Speaker Models

Modelling User Affect and Sentiment in Intelligent User Interfaces

Contact Info

Product

Resources

About