Identifying Human Emotions from Facial Expressions with Deep Learning

Phavish, Babajee; Suddul, Geerish; Armoogum, Sandhya; Foogooa, Ravi

doi:10.1109/zinc50678.2020.9161445

Cited by 32 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Phavish et al explored the classification of human facial expressions through a deeplearning approach using CNN models. The shape and location of the facial composition information were extracted using an edge detection framework and a stochastic classifier was used for facial expression classification [21]. Collecting photo-and video-based facial expression data is expensive, and there are concerns regarding personal information leakage.…”

Section: A Emotion Recognition and Classification Based On The Snsmentioning

confidence: 99%

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Kang

Kim

2022

IEEE Access

View full text Add to dashboard Cite

To apply emotion recognition and classification technology to the field of human-robot interaction, it is necessary to implement fast data processing and model weight reduction. This paper proposes a new photoplethysmogram (PPG) and galvanic skin response (GSR) signals-based labeling method using Asian multimodal data, a real-time emotion classification method, a 1d convolutional neural network autoencoder model, and a lightweight model obtained using knowledge distillation. In addition, the model performance was verified using the public DEAP dataset and the Asian multi-modal dataset 'MERTI-Apps'. For emotion classification, bio-signal data were window-sliced in 1-pulse units, and the label was reset to reflect the characteristics of the PPG and GSR signals. Simple data pre-processing, such as the prevention of loss and waveform duplication, was performed without using handcrafted features. The experiment showed that the accuracy of the proposed model using MERTI-Apps was 79.18% and 74.84% in the case of arousal and valence, respectively, for 3-class criteria, and the accuracy of the proposed model using DEAP was 81.33% and 80.25% in the case of arousal and valence, respectively, for 2-class criteria. The accuracy of the lightweight model was 77.87% and 73.49% in the case of arousal and valence, respectively, for 3-class criteria and its calculation time was reduced by more than 80% compared to the proposed 1d convolutional autoencoder model. We also confirmed that the proposed model improved computational time and accuracy compared to previous studies using MERTI-Apps and the lightweight model used in limited hardware environments enabled fast computation and real-time emotion classification.

show abstract

Section: A Emotion Recognition and Classification Based On The Snsmentioning

confidence: 99%

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Kang

Kim

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This specific area involves systems to classify the fundamental human emotions with current artificial intelligence algorithms, particularly neural networks FACS [20]. The FER general architecture comprises three phases: preprocessing, extraction of features, and classification [14], [21], [22].…”

Section: Facial Expression Recognition (Fer)mentioning

confidence: 99%

Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

Abdullah

Abdulazeez

2021

JSCDM

View full text Add to dashboard Cite

Facial emotional processing is one of the most important activities in effective calculations, engagement with people and computers, machine vision, video game testing, and consumer research. Facial expressions are a form of nonverbal communication, as they reveal a person's inner feelings and emotions. Extensive attention to Facial Expression Recognition (FER) has recently been received as facial expressions are considered. As the fastest communication medium of any kind of information. Facial expression recognition gives a better understanding of a person's thoughts or views and analyzes them with the currently trending deep learning methods. Accuracy rate sharply compared to traditional state-of-the-art systems. This article provides a brief overview of the different FER fields of application and publicly accessible databases used in FER and studies the latest and current reviews in FER using Convolution Neural Network (CNN) algorithms. Finally, it is observed that everyone reached good results, especially in terms of accuracy, with different rates, and using different data sets, which impacts the results.

show abstract

“…Convolutional deep networks based techniques have been commonly used for emotion categorization. Recently, a convolutional neural network (CNN) based system is created to classify six basic plus neutral emotions of the facial action coding system [10]. Initially, a face in an image was detected using the Viola-Jones algorithm [11].…”

Section: Related Workmentioning

confidence: 99%

“…Deep learning algorithms have been used to categorize emotion from images [10], [13], [15]. However, feeding deep models directly with face images considers the color distribution within pixels by representing all pixels in an image to be equally important.…”

Section: Related Workmentioning

confidence: 99%

Lateralized Approach for Robustness Against Attacks in Emotion Categorization from Images

Shehu

Siddique

Browne

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep learning has achieved a high classification accuracy on image classification tasks, including emotion categorization. However, deep learning models are highly vulnerable to adversarial attacks. Even a small change, imperceptible to a human (e.g. one-pixel attack), can decrease the classification accuracy of deep models. One reason could be their homogeneous representation of knowledge that considers all pixels in an image to be equally important is easily fooled. Enabling multiple representations of the same object, e.g. at the constituent and holistic viewpoints provides robustness against attacking a single view. This heterogeneity is provided by lateralization in biological systems. Lateral asymmetry of biological intelligence suggests heterogeneous learning of objects. This heterogeneity allows information to be learned at different levels of abstraction, i.e. at the constituent and the holistic level, enabling multiple representations of the same object. This work aims to create a novel system that can consider heterogeneous features e.g. mouth, eyes, nose, and jaw in a face image for emotion categorization. The experimental results show that the lateralized system successfully considers constituent and holistic features to exhibit robustness to unimportant and irrelevant changes to emotion in an image, demonstrating performance accuracy better than (or similar) to the deep learning system (VGG19). Overall, the novel lateralized method shows a stronger resistance to changes (10.86 − 47.72% decrease) than the deep model (25.15 − 83.43% decrease). The advances arise by allowing heterogeneous features, which enable constituent and holistic representations of image components.

show abstract

Identifying Human Emotions from Facial Expressions with Deep Learning

Cited by 32 publications

References 13 publications

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

1D Convolutional Autoencoder-Based PPG and GSR Signals for Real-Time Emotion Classification

Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

Lateralized Approach for Robustness Against Attacks in Emotion Categorization from Images

Contact Info

Product

Resources

About