Emotion recognition from facial images with arbitrary views

Abstract

Facial expression recognition has been predominantly utilized to analyze the emotional status of human beings. In practice nearly frontal-view facial images may not be available. Therefore, a desirable property of facial expression recognition would allow the user to have any head pose. Some methods on non-frontal-view facial images were recently proposed to recognize the facial expressions by building discriminative subspace in specific views. We argue that this kind of approach ignores (1) the discrimination of inter-class samples with the same view label and (2) the closeness of intra-class samples with all view labels. This paper proposes a new method to recognize arbitrary-view facial expressions by using discriminative neighborhood preserving embedding and multi-view concepts. It first captures the discriminative property of inter-class samples. In addition, it explores the closeness of intra-class samples with arbitrary view in a low-dimensional subspace. Experimental results on BU-3DFE and Multi-PIE databases show that our approach achieves promising results for recognizing facial expressions with arbitrary views.

Facial expression recognition under a wide range of head poses

“…Facial expression analysis methods based only on 2D intensity images can be grouped in view-dependent and view independent approaches [5]. In [10] Zheng et al use regional covariance matrices for view independent facial expression recognition.…”

“…However, the authors show experimental results with only a very limited discrete set of pan rotations and the method needs a (limited) number of non-frontal face training samples. Recently Huang et al [5] proposed to use Multiset 978-1-4799-6026-2/15/$31.00 ©2015 IEEE Canonical Correlation Analysis [15] in order to exploit the correlation between facial expressions and pose labels. They show promising results with a limited set of discretized pan angles.…”

“…However, despite the large applicative interest and the many years of research on this topic, facial expression recognition is still an open problem. One of the main open issues is the drastic drop of accuracy of the existing facial expression approaches when applied to nonfrontal faces [5]. In this paper we propose to transform the problem of emotion recognition with drastic head poses into a missing data classification problem.…”

Pairwise linear regression: An efficient and fast multi-view facial expression recognition

“…Facial expression recognition (FER) could be broadly categorized into the three categories: 1) Geometric-based methods [5], [6], [7], 2) Appearance-based methods [8], [9], [10], [11], [12], and 3) hybrid methods which use both texture and shape information [13]. Recent work on geometric-based method includes regression-based of different mapping functions of geometric features which proposed by Rudovic et al [6] and mapped 2D facial points from nonfrontal to frontal view and then used new mapped points for expression recognition.…”

Mapping Forests: A Comprehensive Approach for Nonlinear Mapping Problems

“…We cropped facial regions using a semiautomatic algorithm into the dimension of 175 × 200 pixels. In order to evaluate our model, we use two protocols: (1) Protocol 1, similar to [18,21] containing 13 viewpoints; and (2) Protocol 2, similar to [25,44] containing 7 viewpoints. Each feature vector has a dimensionality of 14,952, which is reduced to 500 (similar to the BU3DFE features) using PCA.…”