Efficient Facial Feature Learning with Wide Ensemble-Based Convolutional Neural Networks

Abstract

Ensemble methods, traditionally built with independently trained de-correlated models, have proven to be efficient methods for reducing the remaining residual generalization error, which results in robust and accurate methods for real-world applications. In the context of deep learning, however, training an ensemble of deep networks is costly and generates high redundancy which is inefficient. In this paper, we present experiments on Ensembles with Shared Representations (ESRs) based on convolutional networks to demonstrate, quantitatively and qualitatively, their data processing efficiency and scalability to large-scale datasets of facial expressions. We show that redundancy and computational load can be dramatically reduced by varying the branching level of the ESR without loss of diversity and generalization power, which are both important for ensemble performance. Experiments on large-scale datasets suggest that ESRs reduce the remaining residual generalization error on the AffectNet and FER+ datasets, reach human-level performance, and outperform state-of-the-art methods on facial expression recognition in the wild using emotion and affect concepts.

The FaceChannel: A Fast and Furious Deep Neural Network for Facial Expression Recognition

“…We outperform their result by almost 6% as reported in Table 6. We also outperform the results reported in Miao et al [43], Li et al [44], and Siqueira et al [26] which employ different type of complex neural networks to learn facial expressions. Table 7 reports the experiments of the different versions of the FaceChannel on the FABO dataset.…”

“…We observe that the network provides an stable classification over all the classes, including the ones with less examples such as the "Surprise" class. [36] 84.98% SHCNN [43] 86.54 TFE-JL [44] 84.3 ESR-9 [26] 87.15 FaceChannel 90.50% Temporal normalization [45] 66.50 Bag of words [45] 59.00 SVM [46] 32.49 Adaboost [46] 35.22 Face channel 80.54…”

“…Most of these models, however, employ large and deep neural networks that demand a high computational power for training and re-adapting [10,23,24]. As a result, these models specialize on recognizing emotion expressions under conditions represented in the datasets they are trained with [25,26]. Thus, when these models are used to recognize facial expression under different conditions, not represented in the training data, they tend to perform poorly.…”

Multi‐Task and Attention Collaborative Network for Facial Emotion Recognition

“…to perform facial emotion recognition estimation, achieving good results. Siqueira et al [33] uses ensembles with shared representations (ESRs) based on convolutional networks for facial emotion recognition. It can be seen that the RMSE results of ours are better than the Reference [33].…”

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

“…The comparison results of recognition accuracy on the Fer2013, CK+, FER+ and RAF data sets are listed in Table IV, Table V, Table VI and Table VII, respectively. [55] 68.79% DenseNet [25] 71.02% GoogLeNet [25] 65.76% VGG-Face [25] 69.18% Ref [51] 61.86% Ref [29] 65.03% Ref [30] 68% Ref [31] 66% Ref [34] 70.02% AlexNet [42] 66.67% VGGNet [42] 69.41% ResNet [42] 70.74% MBCC-CNN(ours) 71.52% [46] 97.02% Ref [53] 94.82% Ref [23] 90.48% Ref [28] 97.38% Ref [51] 97.35% Ref [33] 94.67% Ref [34] 98% WMCNN-LSTM [36] 97.50% Ref [35] 87.20% Ref [54] 96.46% Ref [37] 98.38% Ref [44] 96.15% Ref [47] 96.46% Ref [61] 96.28% MBCC-CNN(ours) 98.48% TableVI THE ACCURACY COMPARISON RESULTS ON THE FER+ DATA SET Method Accuracy DenseNet [62] 86.54% Ref [63] 85.67% Ref [64] 87.15% Ref [65] 85.10% Ref [66] 82.00% Ref [67] 84.29% Ref [68] 87.76% ResNet50 (transfer learning) [69] 79.90% MBCC-CNN(ours)…”