Identity-Aware Convolutional Neural Network for Facial Expression Recognition

Meng, Zibo; Liu, Ping; Cai, Jie; Han, Shizhong; Tong, Yan

doi:10.1109/fg.2017.140

Cited by 290 publications

(183 citation statements)

References 43 publications

Supporting

Mentioning

177

Contrasting

Unclassified

Order By: Relevance

“…Interestingly, the top scoring system in the 2013 FER Challenge is a deep convolutional neural network [31], while the best handcrafted model ranked only in the fourth place [14]. With only a few exceptions [1,29,30], most of the recent works on facial expression recognition are based on deep learning [2,[6][7][8]11,13,16,[18][19][20][21]23,25,[34][35][36]. Some of these recent works [13,16,18,34,35] proposed to train an ensemble of convolutional neural networks for improved performance, while others [4,15] combined deep features with handcrafted features such as SIFT [22] or Histograms of Oriented Gradients (HOG) [5].…”

Section: Related Artmentioning

confidence: 99%

See 1 more Smart Citation

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Ionescu

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

In this paper, we present an approach based on convolutional neural networks (CNNs) for facial expression recognition in a difficult setting with severe occlusions. More specifically, our task is to recognize the facial expression of a person wearing a virtual reality (VR) headset which essentially occludes the upper part of the face. In order to accurately train neural networks for this setting, in which faces are severely occluded, we modify the training examples by intentionally occluding the upper half of the face. This forces the neural networks to focus on the lower part of the face and to obtain better accuracy rates than models trained on the entire faces. Our empirical results on two benchmark data sets, FER+ and AffectNet, show that our CNN models' predictions on lower-half faces are up to 13% higher than the baseline CNN models trained on entire faces, proving their suitability for the VR setting. Furthermore, our models' predictions on lower-half faces are no more than 10% under the baseline models' predictions on full faces, proving that there are enough clues in the lower part of the face to accurately predict facial expressions.

show abstract

Section: Related Artmentioning

confidence: 99%

“…In the past few years, most works [2,6,8,11,13,16,[18][19][20][21]23,25,31,[34][35][36] have focused on building and training deep neural networks in order to achieve stateof-the-art results. Engineered models based on handcrafted features [1,14,29,30] Fig.…”

Section: Introductionmentioning

confidence: 99%

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Ionescu

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…19 7 Dynamic Cai et al [42] 94. 35 7 Static Meng et al [43] 95.37 7 static li et al [44] 95.78 6 static chu et al [45] 96.40 7 Dynamic Ding et al [4] 96.8 8 Static Mollahosseini et al [10] 97.80 7 Static Zhao et al [46] 97. 30 6 Dynamic Ding et al [4] 98.60 6 Static Jung et al [18] 97.…”

Section: Methodsmentioning

confidence: 99%

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

Otberdout

Kacem

Daoudi

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

In this paper, we propose a new approach for facial expression recognition using deep covariance descriptors. The solution is based on the idea of encoding local and global Deep Convolutional Neural Network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By conducting the classification of static facial expressions using Support Vector Machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that both the proposed static and dynamic approaches achieve state-of-the-art performance for facial expression recognition outperforming many recent approaches.Index Terms-Convolutional neural networks, covariance matrix, deep trajectory, facial expression recognition, symmetric positive definite manifold.

show abstract

“…Many existing techniques target facial expression recognition in images and video sequences [26]. Earlier works on facial expression recognition were concentrated on images [24,22,28,21,15,5,3,2,41]. However, they do not consider temporal variations.…”

Section: Related Workmentioning

confidence: 99%

LBVCNN: Local Binary Volume Convolutional Neural Network for Facial Expression Recognition From Image Sequences

Kumawat

Verma

Raman

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Recognizing facial expressions is one of the central problems in computer vision. Temporal image sequences have useful spatio-temporal features for recognizing expressions. In this paper, we propose a new 3D Convolution Neural Network (CNN) that can be trained end-to-end for facial expression recognition on temporal image sequences without using facial landmarks. More specifically, a novel 3D convolutional layer that we call Local Binary Volume (LBV) layer is proposed. The LBV layer, when used with our newly proposed LBVCNN network, achieve comparable results compared to state-of-the-art landmark-based or without landmark-based models on image sequences from CK+, Oulu-CASIA, and UNBC McMaster shoulder pain datasets. Furthermore, our LBV layer reduces the number of trainable parameters by a significant amount when compared to a conventional 3D convolutional layer. As a matter of fact, when compared to a 3 × 3 × 3 conventional 3D convolutional layer, the LBV layer uses 27 times less trainable parameters.

show abstract

Identity-Aware Convolutional Neural Network for Facial Expression Recognition

Cited by 290 publications

References 43 publications

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

LBVCNN: Local Binary Volume Convolutional Neural Network for Facial Expression Recognition From Image Sequences

Contact Info

Product

Resources

About