ETS System for AV+EC 2015 Challenge

Cardinal, Patrick; Dehak, Najim; Koerich, Alessandro L.; Alam, Jahangir; Boucher, Patrice

doi:10.1145/2808196.2811639

Cited by 16 publications

(16 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though the problem of emotion recognition has been extensively studied in the literature, we only focus on works that predicted dimensional values, continuously in time. Successful attempts to solving the continuous emotion recognition problem relied on DNNs [4], BLSTMs [5], and more commonly, support vector regression (SVR) classifiers [6]. With the exception of BLSTMs, such approaches do not incorporate longterm dependencies unless coupled with feature engineering.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, Valstar et al [2] showed that it was necessary to consider larger windows when making frame-level emotion predictions (four seconds for arousal and six seconds for valence). Le et al [3] and Cardinal et al [4] found that increasing the number of contextual frames when training a deep neural network (DNN) for making frame-level emotion predictions is helpful but only to a certain point. Bidirectional long short-term memory networks (BLSTMs) can naturally incorporate longterm temporal dependencies between features; explaining their success in continuous emotion recognition tasks (e.g., [5]).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Capturing Long-Term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognition

Khorram

Aldeneh

Dimitriadis³

et al. 2017

Interspeech 2017

View full text Add to dashboard Cite

The goal of continuous emotion recognition is to assign an emotion value to every frame in a sequence of acoustic features. We show that incorporating long-term temporal dependencies is critical for continuous emotion recognition tasks. To this end, we first investigate architectures that use dilated convolutions. We show that even though such architectures outperform previously reported systems, the output signals produced from such architectures undergo erratic changes between consecutive time steps. This is inconsistent with the slow moving ground-truth emotion labels that are obtained from human annotators. To deal with this problem, we model a downsampled version of the input signal and then generate the output signal through upsampling. Not only does the resulting downsampling/upsampling network achieve good performance, it also generates smooth output trajectories. Our method yields the best known audioonly performance on the RECOLA dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Capturing Long-Term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognition

Khorram

Aldeneh

Dimitriadis³

et al. 2017

Interspeech 2017

View full text Add to dashboard Cite

show abstract

“…In the last years, several works attempt to predict valence and arousal using machine learning algorithms [5], [10], [19], [1]. Eyben et al [5] proposed a fully automatic audiovisual recognition approach based on Long Short-Term Memory (LSTM) modeling of word-level audio and video features.…”

Section: Introductionmentioning

confidence: 99%

Emotion Recognition Using Fusion of Audio and Video Features

Ortega

Cardinal

Koerich

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

Self Cite

View full text Add to dashboard Cite

In this paper we propose a fusion approach to continuous emotion recognition that combines visual and auditory modalities in their representation spaces to predict the arousal and valence levels. The proposed approach employs a pre-trained convolution neural network and transfer learning to extract features from video frames that capture the emotional content. For the auditory content, a minimalistic set of parameters such as prosodic, excitation, vocal tract, and spectral descriptors are used as features. The fusion of these two modalities is carried out at a feature level, before training a single support vector regressor (SVR) or at a prediction level, after training one SVR for each modality. The proposed approach also includes preprocessing and postprocessing techniques which contribute favorably to improving the concordance correlation coefficient (CCC). Experimental results for predicting spontaneous and natural emotions on the RECOLA dataset have shown that the proposed approach takes advantage of the complementary information of visual and auditory modalities and provides CCCs of 0.749 and 0.565 for arousal and valence, respectively.

show abstract

“…Automatic facial expression recognition has been investigated in the last years due to the great number of applications, ranging from human-computer interaction, emotion analysis [2] to detection of driver fatigue [24]. Several approaches based on handcrafted features [3], [14], [24], [25] such as textural, eigenfaces, etc.…”

Section: Introductionmentioning

confidence: 99%

Memory Integrity of CNNs for Cross-Dataset Facial Expression Recognition

Tannugi

Britto

Koerich

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

View full text Add to dashboard Cite

Facial expression recognition is a major problem in the domain of artificial intelligence. One of the best ways to solve this problem is the use of convolutional neural networks (CNNs). However, a large amount of data is required to train properly these networks but most of the datasets available for facial expression recognition are relatively small. A common way to circumvent the lack of data is to use CNNs trained on large datasets of different domains and fine-tuning the layers of such networks to the target domain. However, the fine-tuning process does not preserve the memory integrity as CNNs have the tendency to forget patterns they have learned. In this paper, we evaluate different strategies of fine-tuning a CNN with the aim of assessing the memory integrity of such strategies in a cross-dataset scenario. A CNN pre-trained on a source dataset is used as the baseline and four adaptation strategies have been evaluated: fine-tuning its fully connected layers; fine-tuning its last convolutional layer and its fully connected layers; retraining the CNN on a target dataset; and the fusion of the source and target datasets and retraining the CNN. Experimental results on four datasets have shown that the fusion of the source and the target datasets provides the best trade-off between accuracy and memory integrity.

show abstract

ETS System for AV+EC 2015 Challenge

Cited by 16 publications

References 8 publications

Capturing Long-Term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognition

Capturing Long-Term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognition

Emotion Recognition Using Fusion of Audio and Video Features

Memory Integrity of CNNs for Cross-Dataset Facial Expression Recognition

Contact Info

Product

Resources

About