Sparse Simultaneous Recurrent Deep Learning for Robust Facial Expression Recognition

Abstract

Facial expression recognition is a challenging task that involves detection and interpretation of complex and subtle changes in facial muscles. Recent advances in feed-forward deep neural networks (DNNs) have offered improved object recognition performance. Sparse feature learning in feed-forward DNN models offers further improvement in performance when compared to the earlier handcrafted techniques. However, the depth of the feed-forward DNNs and the computational complexity of the models increase proportional to the challenges posed by the facial expression recognition problem. The feed-forward DNN architectures do not exploit another important learning paradigm, known as recurrency, which is ubiquitous in the human visual system. Consequently, this paper proposes a novel biologically relevant sparse-deep simultaneous recurrent network (S-DSRN) for robust facial expression recognition. The feature sparsity is obtained by adopting dropout learning in the proposed DSRN as opposed to usual handcrafting of additional penalty terms for the sparse representation of data. Theoretical analysis of S-DSRN shows that the dropout learning offers desirable properties such as sparsity, and prevents the model from overfitting. Experimental results also suggest that the proposed method yields better performance accuracy, requires reduced number of parameters, and offers reduced computational complexity than that of the previously reported state-of-the-art feed-forward DNNs using two of the most widely used publicly available facial expression data sets. Furthermore, we show that by combining the proposed neural architecture with a state-of-the-art metric learning technique significantly improves the overall recognition performance. Finally, a graphical processing unit (GPU)-based implementation of S-DSRN is obtained for real-time applications.

A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

“…This model can mine the interaction context of the face and achieve accurate fine-grained face prediction. In [40], Alam M et al proposed a biologically related sparse deep synchronous recursive network (S-DSRN) for the robust recognition of facial expressions. This method uses dropout learning to obtain feature sparsity, which can provide good classification performance and has a low computational complexity.…”

Boosting the Face Recognition Performance of Ensemble Based LDA for Pose, Non-uniform Illuminations, and Low-Resolution Images

“…Moreover, they project face pairs into the latent feature space to obtain the reduced distance of each positive pair and maximum distance of negative pairs. In [32], authors develop a Sparse-Deep Simultaneous Recurrent Network (S-DSRN) to achieve a robust recognition. The feature sparsity is obtained by adopting dropout learning in the DSRN as opposed to usual handcrafting of additional penalty terms for the sparse representation of data.…”

Patch Attention Layer of Embedding Handcrafted Features in CNN for Facial Expression Recognition

“…FER can also be divided into the traditional method [15,27,[30][31][32]38,40], deep learning method [16][17][18]20,21,[23][24][25][26]35,36,39,[41][42][43], or a combination of the two [11,12,22,28,29,33,37]. Traditional FER systems usually involve facial representation and expression classification.…”

“…In addition, most literatures [12,22,23,33] have used standard convolutional layers, whose weights are shared across a whole face, to learn facial features. However, different regions of an aligned face have different local statistics, and the spatial stationarity assumption of convolution cannot hold [47].…”