Classifying and Visualizing Emotions with Emotional DAN

Abstract

Classification of human emotions remains an important and challenging task for many computer vision algorithms, especially in the era of humanoid robots which coexist with humans in their everyday life. Currently proposed methods for emotion recognition solve this task using multi-layered convolutional networks that do not explicitly infer any facial features in the classification phase. In this work, we postulate a fundamentally different approach to solve emotion recognition task that relies on incorporating facial landmarks as a part of the classification loss function. To that end, we extend a recently proposed Deep Alignment Network (DAN) with a term related to facial features. Thanks to this simple modification, our model called Emotion-alDAN is able to outperform state-of-the-art emotion classification methods on two challenging benchmark dataset by up to 5%. Furthermore, we visualize image regions analyzed by the network when making a decision and the results indicate that our EmotionalDAN model is able to correctly identify facial landmarks responsible for expressing the emotions.

eXnet: An Efficient Approach for Emotion Recognition in the Wild

“…Tautkute et al [10] recently claimed the best results on the CK+ and Indian Spontaneous Expression Database ( ISED). The presented approach used deep align network for landmark detection, which further led to the classification of facial expressions.…”

“…VGG [7] 94.6 ResNet [20] 94 DenseNet [47] 92 DeXpression [13] 96 Tautkute et al [10] 92 Lopes et al [16] 92.73 Jain et al [17] 93.24 Shao et al [18] Table 6. Comparison of eXnet with benchmark networks on the RAF-DB dataset.…”

“…With the recent evolution of deep learning and Convolutional Neural Networks (CNN), various models have shown effective results in the field of computer vision. Most of the crucial advances inspired by AlexNet [6] and VGG [7] have been made [8][9][10][11][12][13][14][15][16][17][18][19], which use end-to-end approaches to classify a given image using facial expression. The typical pattern among these works is the addition of extra layers as well as increments of additional neurons per layer for achieving higher accuracy.…”

Multimodal Emotion Recognition Using Non-Inertial Loss Function

“…Moreover, Action Units (AUs) [2], [3], [4] were proposed to model facial behavior, and the combination of AUs could also be utilized for facial expression recognition. Most studies are based on the seven basic categories [7], [8], [9], [10], [11], [12] with some researchers using triplet expression recognition [13].…”

Using neural and distance-based machine learning techniques in order to identify genuine and acted emotions from facial expressions

“…Intelligence systems are developed with aim on the automation of human tasks. In this context, there are many solutions on the literature for automatic identification of human emotions [2], [3], [4], [5], [6], [7]. On the other hand, there are situations that emotions can be necessarily not genuine, they can be acted, and have few classifiers to differentiate emotions from acted to genuine [8], [9], [10].…”