Incremental approach for multi-modal face expression recognition system using deep neural networks

Abstract

Facial expression recognition (FER) is still one of the most challenging tasks. Convolutional neural network (CNN) and deep convolutional neural network (DCNN) has evolved as an efficient tool for FER models, but they differ significantly in terms of their network configuration and architecture. There exists a variety of bottlenecks in existing FER systems, such as they lack in generalising their algorithms. In this paper, we propose a model based on DCNN to overcome these challenges. Firstly, the proposed model focuses on the selection of an appropriate activation function depending on its accuracy and training loss over a database. Secondly, an incremental strategy is used in which deeper models are developed simultaneously from shallower networks to increase the accuracy with less training loss. Lastly, by an ensemble of CNN and DCNNs, the model achieves an accuracy of 74.15% for FER2013, 96.20% for CK+, and 98.25% for JAFFE databases, outperforming previous work.