Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

Abstract

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition. Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision

Low-resolution facial emotion recognition on low-cost devices

“…Using multiple hyperparameters causes this work to depend on the tuning process, which is expensive. Furthermore, Bodavarapu and Srinivas [17] has designed a novel CNN model (FERConvNet) with small dimensional inputs. It employs a 2D-convolutional layer followed by batch normalization and dropout to inhibit overfitting problems.…”

Smart Classroom Monitoring Using Novel Real-Time Facial Expression Recognition System

“…The research argues that analyzing facial expressions using a huge imagery dataset with unnecessary input dimensions can decrease the efficiency of the FER system. Similarly, another study concludes that images taken from different angles, low resolution, and noisy backgrounds can be problematic in automatic facial expression recognition [29][30][31][32][33]. Another research has argued that static images are not sufficient for automatic facial expression recognition and the authors conducted a study using recorded video of the classroom to improve the accuracy of FER [34][35][36][37].…”

Facial Emotion Detection using Convolutional Neural Network

“…A new approach is shown, which leads to a progressive increase in the model's accuracy by first tweaking each of the pre-trained DCNN blocks individually, after training the dense layer(s). Eight alternative pre-trained DCNN models (VGG-16, VGG-19, ResNet-18, ResNet-34, ResNet-50, ResNet-152, Inception-v3, and DenseNet-161) as well as the well-known KDEF and JAFFE facial image datasets are used to verify the proposed FER system [16].…”