High-Level Geometry-based Features of Video Modality for Emotion Prediction

Weber, Raphaël; Barrielle, Vincent; Soladié, Catherine; Séguier, Renaud

doi:10.1145/2988257.2988262

Cited by 15 publications

(11 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The objective is to communicate and interpret emotions through expressions using multiple modalities. Various methods have been proposed for depression analysis [11], [19], [20], including most recent works from AVEC2016 [21]- [23].…”

mentioning

confidence: 99%

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Jan

Meng

Gaus

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

158

View full text Add to dashboard Cite

A human being's cognitive system can be simulated by artificial intelligent systems. Machines and robots equipped with cognitive capability can automatically recognize a humans mental state through their gestures and facial expressions. In this paper, an artificial intelligent system is proposed to monitor depression. It can predict the scales of Beck depression inventory II (BDI-II) from vocal and visual expressions. First, different visual features are extracted from facial expression images. Deep learning method is utilized to extract key visual features from the facial expression frames. Second, spectral lowlevel descriptors and mel-frequency cepstral coefficients features are extracted from short audio segments to capture the vocal expressions. Third, feature dynamic history histogram (FDHH) is proposed to capture the temporal movement on the feature space. Finally, these FDHH and audio features are fused using regression techniques for the prediction of the BDI-II scales. The proposed method has been tested on the public Audio/Visual Emotion Challenges 2014 dataset as it is tuned to be more focused on the study of depression. The results outperform all the other existing methods on the same dataset.

show abstract

mentioning

confidence: 99%

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Jan

Meng

Gaus

et al. 2018

IEEE Trans. Cogn. Dev. Syst.

158

View full text Add to dashboard Cite

show abstract

“…In the state-of-the-art, researchers in [22] tested a deep-learning model which consists of RNN and CNN which showed a Concordance Correlation Coefficient (CCC) [44] of 0.10 on the arousal dimension and 0.33 on the valence dimension based on EDA only. They used AVEC 2016 dataset [23,24].…”

Section: Discussionmentioning

confidence: 99%

“…In [22], authors used the AVEC 2016 dataset [23,24], they proposed a deep-learning model that consists of a CNN followed by a recurrent neural network and then fully connected layers. They showed that an end-to-end deep-learning approach directly depending on raw signals can replace feature engineering for emotion recognition purposes.…”

Section: Related Workmentioning

confidence: 99%

A Deep-Learning Model for Subject-Independent Human Emotion Recognition Using Electrodermal Activity Sensors

Machot

Elmachot

Ali³

et al. 2019

Sensors

View full text Add to dashboard Cite

One of the main objectives of Active and Assisted Living (AAL) environments is to ensure that elderly and/or disabled people perform/live well in their immediate environments; this can be monitored by among others the recognition of emotions based on non-highly intrusive sensors such as Electrodermal Activity (EDA) sensors. However, designing a learning system or building a machine-learning model to recognize human emotions while training the system on a specific group of persons and testing the system on a totally a new group of persons is still a serious challenge in the field, as it is possible that the second testing group of persons may have different emotion patterns. Accordingly, the purpose of this paper is to contribute to the field of human emotion recognition by proposing a Convolutional Neural Network (CNN) architecture which ensures promising robustness-related results for both subject-dependent and subject-independent human emotion recognition. The CNN model has been trained using a grid search technique which is a model hyperparameter optimization technique to fine-tune the parameters of the proposed CNN architecture. The overall concept’s performance is validated and stress-tested by using MAHNOB and DEAP datasets. The results demonstrate a promising robustness improvement regarding various evaluation metrics. We could increase the accuracy for subject-independent classification to 78% and 82% for MAHNOB and DEAP respectively and to 81% and 85% subject-dependent classification for MAHNOB and DEAP respectively (4 classes/labels). The work shows clearly that while using solely the non-intrusive EDA sensors a robust classification of human emotion is possible even without involving additional/other physiological signals.

show abstract

“…The overview of the three best performing systems of the last two editions of the AVEC challenge shows that, hand-crafted features computed from the ECG clearly outperform those extracted from the EDA for both arousal and valence. The best performance achieved on arousal has been obtained with the system developed by Weber et al [21], using the baseline feature set of the challenge; 19 features composed of linear and non-linear descriptors computed on a band-pass filtered version of the ECG signal. A Support Vector Regression (SVR) model was trained for each subject of the database, and fusion of these single-speaker-regression-models was performed by a linear regression.…”

Section: Related Workmentioning

confidence: 99%

End-to-end learning for dimensional emotion recognition from physiological signals

Keren

Kirschstein

Marchi

et al. 2017

2017 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Dimensional emotion recognition from physiological signals is a highly challenging task. Common methods rely on hand-crafted features that do not yet provide the performance necessary for real-life application. In this work, we exploit a series of convolutional and recurrent neural networks to predict affect from physiological signals, such as electrocardiogram and electrodermal activity, directly from the raw time representation. The motivation behind this socalled end-to-end approach is that, ultimately, the network learns an intermediate representation of the physiological signals that better suits the task at hand. Experimental evaluations show that, this very first study on end-to-end learning of emotion based on physiology, yields significantly better performance in comparison to existing work on the challenging RECOLA database, which includes fully spontaneous affective behaviors displayed during naturalistic interactions. Furthermore, we gain better understanding of the models' inner representations, by demonstrating that some cells' activations in the convolutional network are correlated to a large extent with hand-crafted features.

show abstract

High-Level Geometry-based Features of Video Modality for Emotion Prediction

Cited by 15 publications

References 32 publications

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

A Deep-Learning Model for Subject-Independent Human Emotion Recognition Using Electrodermal Activity Sensors

End-to-end learning for dimensional emotion recognition from physiological signals

Contact Info

Product

Resources

About