Prediction-based learning for continuous emotion recognition in speech

Ringeval, Fabien; Schuller, Björn

doi:10.1109/icassp.2017.7953109

Cited by 44 publications

(31 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These positive PCCs suggest that the difficulty information can help improve the model performance in the learning process. This conclusion confirms our previous findings in [39]. Note that, in [39], the selected database has subjects that are different from the one in this article.…”

Section: Emotion Prediction With Dynamic Difficulty Awareness Traisupporting

confidence: 92%

“…This conclusion confirms our previous findings in [39]. Note that, in [39], the selected database has subjects that are different from the one in this article. Similar observations can be found when calculating the PCCs between the PU and the performance improvement [i. e., PCC(µ, ∆ c ), as shown in the second three rows].…”

Section: Emotion Prediction With Dynamic Difficulty Awareness Traisupporting

confidence: 92%

See 1 more Smart Citation

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Zhang

Han

Coutinho

et al. 2019

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Time-continuous emotion prediction has become an increasingly compelling task in machine learning. Considerable efforts have been made to advance the performance of these systems. Nonetheless, the main focus has been the development of more sophisticated models and the incorporation of different expressive modalities (e. g., speech, face, and physiology). In this paper, motivated by the benefit of difficulty awareness in a human learning procedure, we propose a novel machine learning framework, namely, Dynamic Difficulty Awareness Training (DDAT), which sheds fresh light on the research -directly exploiting the difficulties in learning to boost the machine learning process. The DDAT framework consists of two stages: information retrieval and information exploitation. In the first stage, we make use of the reconstruction error of input features or the annotation uncertainty to estimate the difficulty of learning specific information. The obtained difficulty level is then used in tandem with original features to update the model input in a second learning stage with the expectation that the model can learn to focus on high difficulty regions of the learning process. We perform extensive experiments on a benchmark database (RECOLA) to evaluate the effectiveness of the proposed framework. The experimental results show that our approach outperforms related baselines as well as other well-established time-continuous emotion prediction systems, which suggests that dynamically integrating the difficulty information for neural networks can help enhance the learning process.

show abstract

Section: Emotion Prediction With Dynamic Difficulty Awareness Traisupporting

confidence: 92%

Section: Emotion Prediction With Dynamic Difficulty Awareness Traisupporting

confidence: 92%

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Zhang

Han

Coutinho

et al. 2019

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…Apart from these works that intend to explore innovative neural network architectures for SER, various advanced training strategies have been investigated as well. Furthermore, prediction-based learning was introduced in [7] to incorporate the strength of different models, and the reconstruction-error based learning was exam-ined in [8] to compensate the weakness of a neural network itself. In [9], CNNs and LSTM-RNNs were sophisticatedly concatenated into a joint framework, and trained in an end-to-end manner, i. e., by directly learning a suitable representation of the raw signal.…”

Section: Introductionmentioning

confidence: 99%

“…Whereas the presented conditional adversarial training framework utilises a cascaded structure, as used in our previous work, i. e., prediction-based learning [7] and reconstruction-error-based learning [8], it includes some specific advantages in comparison to those two approaches. The main idea of prediction-based learning is to take advantage of different models where predictions made by a first model are combined with the original features to learn a second model.…”

Section: Introductionmentioning

confidence: 99%

Towards Conditional Adversarial Training for Predicting Emotions from Speech

Ren

Ringeval

Schuller

2018

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Motivated by the encouraging results recently obtained by generative adversarial networks in various image processing tasks, we propose a conditional adversarial training framework to predict dimensional representations of emotion, i. e., arousal and valence, from speech signals. The framework consists of two networks, trained in an adversarial manner: The first network tries to predict emotion from acoustic features, while the second network aims at distinguishing between the predictions provided by the first network and the emotion labels from the database using the acoustic features as conditional information. We evaluate the performance of the proposed conditional adversarial training framework on the widely used emotion database RECOLA. Experimental results show that the proposed training strategy outperforms the conventional training method, and is comparable with, or even superior to other recently reported approaches, including deep and end-to-end learning.

show abstract

“…An extrovert is easy to share and express his emotion and be more social. Introverted people are difficult to express and disguise their sentiments making them socially reserved [2] [3].…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition of Sanskrit Language using Machine Learning

Kakodkar¹,

Borkar²

2018

IJCA

View full text Add to dashboard Cite

A modern development in technology is Speech Emotion Recognition (SER). SER in partnership with HumaneMachine interaction (HMI) has advanced machine intelligence. An emotion precise HMI is designed by integrating speech processing and machine learning algorithm which is sculpted to formulate an automated smart and secure application for detecting emotions in a household as well as in commercial application. This project presents a study of distinguishing emotions by acoustic speech recognition (ASR) using K-means nearest neighbor (K-NN), a machine learning (ML) technique. The most significant paralinguistic information obtained from spectral features is presented by ASR i.e. by using Mel frequency cepstrum coefficient (MFCC). The most important processing techniques methods include feature extraction, feature selection, and classification of emotions. A customized dataset consisting of speech corpus, simulated emotion samples in the Sanskrit language is used to classify emotions in different emotional classes i.e. happy, sad, excitement, fear, anger and disgust. The emotions are classified using a K-NN algorithm over 2 separate models, based on the soft and high pitch voice. Model 1 and 2 achieved about 72.95% and 76.96% recognition rates respectively.

show abstract

Prediction-based learning for continuous emotion recognition in speech

Cited by 44 publications

References 22 publications

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Dynamic Difficulty Awareness Training for Continuous Emotion Prediction

Towards Conditional Adversarial Training for Predicting Emotions from Speech

Speech Emotion Recognition of Sanskrit Language using Machine Learning

Contact Info

Product

Resources

About