MRMR-based ensemble pruning for facial expression recognition

Li, Danyang; Wen, Gongjian

doi:10.1007/s11042-017-5105-z

Cited by 27 publications

(20 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With only a few exceptions [1,32,33], most of the recent works on facial expression recognition are based on deep learning [2,9,10,13,14,17,21,22,24,23,26,28,38,39,40]. Some of these recent works [14,17,21,38,39] proposed to train an ensemble of convolutional neural networks for improved performance, while others [6,16] combined deep features with handcrafted features such as SIFT [25] or Histograms of Oriented Gradients (HOG) [8]. While most works studied facial expression recognition from static images, some works tackled facial expression recognition in video [13,16].…”

Section: Related Workmentioning

confidence: 99%

“…Table 1 includes the results of our combined models, one based on global SVM and another based on local SVM, on Table 1: Results on the FER 2013 [11], the FER+ [2] and the AffectNet [27] data sets. Our combination based on pre-trained, fine-tuned and handcrafted models, with and without data augmentation (aug.), are compared with several state-of-the-art approaches [2,6,14,15,17,21,23,27,34,39,40], which are listed in temporal order. The best result on each data set is highlighted in bold.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…We report results with and without data augmentation. Our models are compared with several state-of-the-art approaches [2,6,14,15,17,21,23,27,34,39,40].…”

Section: Implementation Detailsmentioning

confidence: 99%

“…In the past few years, most works [2,9,10,13,14,17,21,22,24,23,26,28,34,38,39,40] have focused on building and training deep neural networks in order to achieve state-of-the-art results. Engineered models based on handcrafted features [1,15,32,33] have drawn very little attention, since such models usually yield less accurate results compared to deep learning models.…”

Section: Introductionmentioning

confidence: 99%

“…For the combined models, we explore only global or local SVM alternatives. We perform a thorough experimental study on the 2013 Facial Expression Recognition (FER) Challenge data set [11], the FER+ data set [2], and the Affect-Net [27] data set, comparing our combined deep and handcrafted models with recent and relevant state-of-the-art approaches [2,6,14,15,17,21,23,27,34,39,40]. We report top results on each and every data set with our combination of automatic and handcrafted features, especially when local SVM is employed in the learning phase.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

2019

View full text Add to dashboard Cite

We present an approach that combines automatic features learned by convolutional neural networks (CNN) and handcrafted features computed by the bag-of-visual-words (BOVW) model in order to achieve state-of-the-art results in facial expression recognition. To obtain automatic features, we experiment with multiple CNN architectures, pretrained models and training procedures, e.g. Dense-Sparse-Dense. After fusing the two types of features, we employ a local learning framework to predict the class label for each test image. The local learning framework is based on three steps. First, a k-nearest neighbors model is applied in order to select the nearest training samples for an input test image. Second, a one-versus-all Support Vector Machines (SVM) classifier is trained on the selected training samples. Finally, the SVM classifier is used to predict the class label only for the test image it was trained for. Although we have used local learning in combination with handcrafted features in our previous work, to the best of our knowledge, local learning has never been employed in combination with deep features. The experiments on the 2013 Facial Expression Recognition (FER) Challenge data set, the FER+ data set and the AffectNet data set demonstrate that our approach achieves state-of-the-art results. With a top accuracy of 75.42% on FER 2013, 87.76% on the FER+, 59.58% on AffectNet 8-way classification and 63.31% on AffectNet 7-way classification, we surpass the state-of-theart methods by more than 1% on all data sets.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

“…We report results with and without data augmentation. Our models are compared with several state-of-the-art approaches [2,6,14,15,17,21,23,27,34,39,40].…”

Section: Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

2019

View full text Add to dashboard Cite

show abstract

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Ionescu

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

In this paper, we present an approach based on convolutional neural networks (CNNs) for facial expression recognition in a difficult setting with severe occlusions. More specifically, our task is to recognize the facial expression of a person wearing a virtual reality (VR) headset which essentially occludes the upper part of the face. In order to accurately train neural networks for this setting, in which faces are severely occluded, we modify the training examples by intentionally occluding the upper half of the face. This forces the neural networks to focus on the lower part of the face and to obtain better accuracy rates than models trained on the entire faces. Our empirical results on two benchmark data sets, FER+ and AffectNet, show that our CNN models' predictions on lower-half faces are up to 13% higher than the baseline CNN models trained on entire faces, proving their suitability for the VR setting. Furthermore, our models' predictions on lower-half faces are no more than 10% under the baseline models' predictions on full faces, proving that there are enough clues in the lower part of the face to accurately predict facial expressions.

show abstract

Teacher–student training and triplet loss to reduce the effect of drastic face occlusion

Duta

Ionescu

2021

Machine Vision and Applications

View full text Add to dashboard Cite

We study a series of recognition tasks in two realistic scenarios requiring the analysis of faces under strong occlusion. On the one hand, we aim to recognize facial expressions of people wearing virtual reality headsets. On the other hand, we aim to estimate the age and identify the gender of people wearing surgical masks. For all these tasks, the common ground is that half of the face is occluded. In this challenging setting, we show that convolutional neural networks trained on fully visible faces exhibit very low performance levels. While fine-tuning the deep learning models on occluded faces is extremely useful, we show that additional performance gains can be obtained by distilling knowledge from models trained on fully visible faces. To this end, we study two knowledge distillation methods, one based on teacher–student training and one based on triplet loss. Our main contribution consists in a novel approach for knowledge distillation based on triplet loss, which generalizes across models and tasks. Furthermore, we consider combining distilled models learned through conventional teacher–student training or through our novel teacher–student training based on triplet loss. We provide empirical evidence showing that, in most cases, both individual and combined knowledge distillation methods bring statistically significant performance improvements. We conduct experiments with three different neural models (VGG-f, VGG-face and ResNet-50) on various tasks (facial expression recognition, gender recognition, age estimation), showing consistent improvements regardless of the model or task.

show abstract

MRMR-based ensemble pruning for facial expression recognition

Cited by 27 publications

References 42 publications

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Teacher–student training and triplet loss to reduce the effect of drastic face occlusion

Contact Info

Product

Resources

About