Real Time Face Expression Recognition along with Balanced FER2013 Dataset using CycleGAN

Abstract

Human face expression recognition is an active research area that has massive applications in medical field, crime investigation, marketing, online learning, automobile safety and video games. The first part of this research defines a deep neural network model-based framework for recognizing the seven main types of facial expression, which are found in all cultures. The proposed methodology involves four stages: (a) preprocessing the FER2013 dataset through relabeling to avoid misleading results and getting rid of non-face and non-frontal faces; (b) design of an efficient stable Cycle Generative Adversarial Network (CycleGAN), which provides unsupervised expression-to-expression translation. The CycleGAN has been designed and trained with a new cycle consistency loss. (c) Generating new images to overcome the class imbalance and finally (d) building the DNN architecture for recognizing the face sign expression, using the pretrained VGG-Face model with vggface weights. The second part encompasses the design of a GPU-accelerated face expression recognition system for real time video sequences using NVIDIAs Compute Unified Device Architecture (CUDA). OpenCV library has been compiled from scratch with CUDA and NVIDIA CUDA Deep Neural Network library, cuDNN. For face detection stage Haar Cascaded and deep learning were used and tested using both CPU and GPU as backend. Results show that the designed model run time to recognize a face sign is 0.44 seconds. Besides, the average test accuracy has been increased from 64% for the original FER2013 dataset to 91.76% for the modified balanced version using the same transfer learning model.

Emotio-Intelligent: A New Adaptive Approach for Intelligent Evacuation in Crisis Situations.

“…A balanced CNN-LSTM [25] is designed and trained. A new deep neural network architecture for recognizing the face sign expression, using the pretrained MobileNetv2 [10] model images weights and the modified balanced version of the FER2013 dataset is designed and implemented.…”

A Lightweight Facial Emotion Recognition System Using Partial Transfer Learning for Visually Impaired People

“…FER2013 is a diverse dataset of images with faces representing seven different emotions happy, angry, sad, fear, surprise, disgust, neutral as well as non-facial and text images. Additionally, there are a total of images with noisy input (sleepy faces) and missing labels [25], [28]. Furthermore, FER2013 exhibits greater diversity including images with facial occlusion, partial faces, and faces with eyeglasses.…”

“…However, FER2013 suffers from imbalanced classes with the following distribution happy: 25%, sad: 16.9%, angry: 13.8%, surprised: 11.2%, fear: 14.3%, disgust: 1.5%. In order to overcome class imbalance issue with additional mislabeled images, an enhanced version of FER2013 (EFER2013) [25] dataset was used which an As the proposed model aims to classify the facial image into six emotions, anger, fear, disgust, happy, sad and surprise, the neutral class samples were omitted and 24,839 samples were used.…”

Role of Zoning in Facial Expression Using Deep Learning

“…The method is simple, providing a high recognition rate and speed of facial expressions on the Original FER 2013 dataset. The same can be used in video sequences, and online learning platforms [29]. A hybrid deep learning model to classify human emotions using DNN and transfer learning is proposed by [30].…”