A deep-learning-based facial expression recognition method using textural features

Mukhopadhyay, Moutan; Dey, Aniruddha; Kahali, Sayan

doi:10.1007/s00521-022-08005-7

Cited by 21 publications

(3 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CNNs method trained on images from prolonged datasets of JAFEE, Cohn-Kanade (CK +) and FER2013 changed into LTP, LBP, and CLBP image features. In [12], an effective FER was developed by employing a new DL Neural Networkregression activation (DR) classification algorithm. Primarily, the Gamma-HE model was used for pre-processing, and then facial facts were removed employing the Pyramid HOG (PHOG) based Supervised Descent (SMD) model.…”

Section: Literature Reviewmentioning

confidence: 99%

Bioinspired Image Processing Enabled Facial Emotion Recognition Using Equilibrium Optimizer With a Hybrid Deep Learning Model

Alzahrani

2024

IEEE Access

View full text Add to dashboard Cite

Owing to the unpredictable nature of human facial expression, Facial emotion recognition (FER) from facial images becomes a tedious process. FER is a field within artificial intelligence (AI) and computer vision (CV) that concentrates on developing algorithms and technologies to interpret and analyze emotional expressions shown on human faces. The major intention of this area of research is to enable computers and machines to automatically classify and detect emotions, including sadness, happiness, anger, etc., based on facial expressions and cues. Deep learning (DL) systems namely recurrent neural network (RNN) and convolutional neural network (CNN) can be used for the task of automatically classifying and detecting emotions from human facial expressions. Furthermore, Bioinspired Image Processing Enabled FER combines principles from biological systems with image processing techniques to better the robustness and accuracy of FER. This technique is inspired by natural processes to enhance the interpretation and understanding of human emotion conveyed by facial expressions. This study designs a new Bioinspired Image Processing Enabled Facial Emotion Recognition using an Equilibrium Optimizer with Hybrid Deep Learning (BIPFER-EOHDL) model. The primary objectives of the BIPFER-EOHDL technique lie in the effectual and automated identification of facial expressions using a hyperparameter-tuned DL algorithm. In the presented BIPFER-EOHDL technique, the median filtering (MF) approach can be applied for image preprocessing. Besides, the BIPFER-EOHDL method applies the EfficientNetB7 model for the process of feature extraction. Meanwhile, the hyperparameter selection of the EfficientNetB7 model takes place by the use of the EO algorithm. Finally, the multi-head attention bi-directional long short-term memory (MA-BLSTM) model is exploited for the recognition and classification of facial emotions. A wide-ranging simulation analysis was performed to validate the higher FER outcomes of the BIPFER-EOHDL methodology. The simulation results stated that the BIPFER-EOHDL technique accomplishes better FER results than other recent approaches.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Bioinspired Image Processing Enabled Facial Emotion Recognition Using Equilibrium Optimizer With a Hybrid Deep Learning Model

Alzahrani

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The MiniExpNet network is used, and its performance is enhanced by incorporating an attention block and a self‐distillation mechanism. The study in [15] develops a convolutional neural network (CNN) model to recognize FEs by taking advantage of textural cues, which are closely linked with changes in FEs. The combination of completed local binary pattern (CLBP) and CNN has shown better recognition rates when applied on benchmark datasets.…”

Section: Related Workmentioning

confidence: 99%

Modelling appearance variations in expressive and neutral face image for automatic facial expression recognition

Kumar H N,

M S,

Asif Shah

et al. 2024

IET Image Processing

View full text Add to dashboard Cite

In automatic facial expression recognition (AFER) systems, modelling the spatio‐temporal feature information in a specific manner, coalescing, and its effective utilization is challenging. The state‐of‐the‐art studies have examined integrating multiple features to enhance the recognition rate of AFER systems. However, the feature variations between expressive and neutral face images are not fully explored to identify the expression class. The proposed research presents an innovative approach to AFER by modelling appearance variations in both expressive and neutral face images. The prominent contributions of the work are developing a novel and hybrid feature space by integrating the discriminative feature distribution derived from expressive and neutral face images; preserving the highly discriminative latent feature distribution using autoencoders. Local binary pattern (LBP) and histogram of oriented gradients (HOG) are the feature descriptors employed to derive the discriminative texture and shape information, respectively. The component‐based approach is employed, wherein the features are derived from the salient facial regions instead of the whole face. The three‐stage stacked deep convolutional autoencoder (SDCA) and multi‐class support vector machine (MSVM) are employed to address dimensionality reduction and classification, respectively. The efficacy of the proposed model is substantiated by empirical findings, which establish its superiority in terms of accuracy in AFER tasks on widely recognized benchmark datasets.

show abstract

“…Accuracy (%) LBP-CNN [64] 79.5 Broad learning [65] 81 STF + LSTM [66] 82 ResNet150 [67] 89 LTP-CNN [64] 89.2 CLBP-CNN [64] 91 LBP+ORB features [68] 93.2 Deep Features + HOG [69] 94.17 Multiple attention network [70] 94.51 hybrid CNN model [71] 94.82 EmoNeXt-Tiny 95.1 Pre-trained VGG19 [72] 96 WMCNN-LSTM [73] 97.5 Pre-trained Resnet50 [72] 97.7 EmoNeXt-Small 97.78 Dynamic cascaded classifier [74] 97.84 DeepEmotion [59] 98 EmoNeXt-Base 98.44 MBCC-CNN [75] 98.48 Pre-trained MobileNet [72] 98.5 Tripe-Structure Network [26] 99.04 Inception-V3-based approach [34] 99.57 ViT + SE [76] 99.8 EfficientNet-XGBoost [37] 100 EmoNeXt-Large 100 EmoNeXt-XLarge 100…”

Section: Modelmentioning

confidence: 99%

A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s disease

Bohi,

Boudouri,

Sfeir

2024

Preprint

View full text Add to dashboard Cite

Facial expressions play a critical role in human communication by effectively conveying a wide range of emotions. Recent advancements in artificial intelligence and computer vision have made deep neural networks powerful tools for recognizing facial emotions. In a prior study, we introduced EmoNeXt, an innovative deep learning framework for facial expression recognition based on a modified ConvNeXt architecture, incorporating several significant enhancements. Initially, we evaluated its performance on the FER2013 dataset. This paper extends the evaluation of EmoNeXt to additional well-known benchmark datasets, including AffectNet and CK+, to further validate its robustness and generalizability. EmoNeXt integrates enhancements such as Spatial Transformer Networks, which help the model focus on areas of the face rich in expressive features, Squeeze-and-Excitation blocks, which improve the model’s ability to understand dependencies between different channels, and a self-attention regularization term that encourages the model to produce compact and effective feature vectors. Furthermore, this paper presents an extensive ablation study conducted on the three datasets, analyzing and confirming the contribution of each added module of our EmoNeXt model. Finally, we explore EmoNeXt’s application in recognizing emotions in elderly individuals with Alzheimer’s disease, addressing the urgent need for accurate emotion recognition to improve patient care.

show abstract

A deep-learning-based facial expression recognition method using textural features

Cited by 21 publications

References 41 publications

Bioinspired Image Processing Enabled Facial Emotion Recognition Using Equilibrium Optimizer With a Hybrid Deep Learning Model

Bioinspired Image Processing Enabled Facial Emotion Recognition Using Equilibrium Optimizer With a Hybrid Deep Learning Model

Modelling appearance variations in expressive and neutral face image for automatic facial expression recognition

A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s disease

Contact Info

Product

Resources

About