Emotion Recognition for Partial Faces Using a Feature Vector Technique

Khoeun, Ratanak; Chophuk, Ponlawat; Chinnasarn, Krisana

doi:10.3390/s22124633

Cited by 9 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the top features of the face are affected by information spreading from the bottom area of the face, we concentrated on the eye and eyebrow regions. For creating top face points and feature extractions, we followed the work of Khoeun et al [ 45 ]. An illustration of the facial emotion recognition approach from the bottom part of the face-masked images is shown in Figure 7 .…”

Section: Methodsmentioning

confidence: 99%

“…We set out to solve the problem of obstructed lower facial features by creating a fast facial landmark detector. We found that during emotional expression, the uncovered areas of the face (the eyes and eyebrows) became wider in contrast to the obscured areas (the lips, cheeks, and nose) [ 45 ]. To further aid in emotion classification, we aimed to implement a landmark identifier and select the face details vectors that indicate the crucial connections among those areas.…”

Section: Methodsmentioning

confidence: 99%

“…Recently, Ngos et al [ 44 ] introduced a graph convolutional neural network utilizing facial landmark features to identify points, and the edges of the graph were constructed by employing the Delaunay technique. Khoeun et al [ 45 ] proposed a feature vector approach for recognizing emotions of masked faces with three key components. The authors used facial landmark identification to retrieve the characteristics of covered faces with masks, and upper facial landmark coordinates were used to identify facial expressions.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People

Mukhiddinov

Djuraev

Akhmedov

et al. 2023

Sensors

View full text Add to dashboard Cite

Current artificial intelligence systems for determining a person’s emotions rely heavily on lip and mouth movement and other facial features such as eyebrows, eyes, and the forehead. Furthermore, low-light images are typically classified incorrectly because of the dark region around the eyes and eyebrows. In this work, we propose a facial emotion recognition method for masked facial images using low-light image enhancement and feature analysis of the upper features of the face with a convolutional neural network. The proposed approach employs the AffectNet image dataset, which includes eight types of facial expressions and 420,299 images. Initially, the facial input image’s lower parts are covered behind a synthetic mask. Boundary and regional representation methods are used to indicate the head and upper features of the face. Secondly, we effectively adopt a facial landmark detection method-based feature extraction strategy using the partially covered masked face’s features. Finally, the features, the coordinates of the landmarks that have been identified, and the histograms of the oriented gradients are then incorporated into the classification procedure using a convolutional neural network. An experimental evaluation shows that the proposed method surpasses others by achieving an accuracy of 69.3% on the AffectNet dataset.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People

Mukhiddinov

Djuraev

Akhmedov

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Khoeun et al proposed a feature vector technique consisting of three steps for recognizing emotions in mask images [12]. First, a mask was applied by using the boundary and area expression techniques such that only the upper part of the image, including the eyes, eyebrows, part of the bridge of the nose, and forehead, was visible.…”

Section: Khoeun Et Al [12]mentioning

confidence: 99%

“…Finally, these features, including the positions of the detected landmarks and histograms of the oriented gradients, were introduced into the classification process by adopting convolutional neural networks (CNNs) and long short-term memory networks. Accuracies of 99.30% and 95.58% were achieved for CK+ and RAF-DB, respectively [12]. Khoeun's research focused on using the upper part of the face to improve facial expression recognition accuracy in the context of the COVID-19 pandemic.…”

Section: Khoeun Et Al [12]mentioning

confidence: 99%

Comparative Analysis of AI-Based Facial Identification and Expression Recognition Using Upper and Lower Facial Regions

Kim

Lee

2023

Applied Sciences

View full text Add to dashboard Cite

The COVID-19 pandemic has significantly impacted society, having led to a lack of social skills in children who became used to interacting with others while wearing masks. To analyze this issue, we investigated the effects of masks on face identification and facial expression recognition, using deep learning models for these operations. The results showed that when using the upper or lower facial regions for face identification, the upper facial region allowed for an accuracy of 81.36%, and the lower facial region allowed for an accuracy of 55.52%. Regarding facial expression recognition, the upper facial region allowed for an accuracy of 39% compared to 49% for the lower facial region. Furthermore, our analysis was conducted for a number of facial expressions, and specific emotions such as happiness and contempt were difficult to distinguish using only the upper facial region. Because this study used a model trained on data generated from human labeling, it is assumed that the effects on humans would be similar. Therefore, this study is significant because it provides engineering evidence of a decline in facial expression recognition; however, wearing masks does not cause difficulties in identification.

show abstract

Improving facial expression recognition for autism with IDenseNet‐RCAformer under occlusions

Selvi,

Parvathy

2024

Intl J of Devlp Neuroscience

View full text Add to dashboard Cite

The term ‘autism spectrum disorder’ describes a neurodevelopmental illness typified by verbal and nonverbal interaction impairments, repetitive behaviour patterns and poor social interaction. Understanding mental states from FEs is crucial for interpersonal interaction and social interaction. But when there are occlusions like glasses, facial hair or self‐occlusion, it becomes harder to identify facial expressions accurately. This research tackles the issue of identifying facial expressions when parts of the face are occluded and suggests an innovative technique to tackle this difficulty. Creating a strong framework for facial expression recognition (FER) that better handles occlusions and increases recognition accuracy is the goal of this research. Therefore, we propose novel Improved DenseNet‐based Residual Cross‐Attention Transformer (IDenseNet‐RCAformer) system to tackle the partial occlusion FER problem in autism patients. The recognition framework's efficacy is assessed using four datasets of facial expressions, and some preprocessing procedures are conducted to increase the expression recognition efficiency. After that, when recognizing expressions, a simple argmax function is applied to get a forecasted landmark position from a heatmap. Then feature extraction phase, local and global representation are captured from preprocessed images by adopting Inception‐ResNet‐V2 approach, Cross‐Attention Transformer, respectively. Moreover, both features are fused by employing the FusionNet method, thereby enhancing system's training speed and precision. After the features are extracted, an improved DenseNet mechanism is applied to efficiently recognize some variety of facial expressions in partially occluded autism patients. A number of performance metrics are determined and analysed to demonstrate the proposed approach's effectiveness, where the IDenseNet‐RCAformer performs best with an accuracy of 98.95%. According to the experimental findings, the proposed framework significantly outperforms the prior recognition frameworks in terms of recognition outcomes.

show abstract

Emotion Recognition for Partial Faces Using a Feature Vector Technique

Cited by 9 publications

References 34 publications

Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People

Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People

Comparative Analysis of AI-Based Facial Identification and Expression Recognition Using Upper and Lower Facial Regions

Improving facial expression recognition for autism with IDenseNet‐RCAformer under occlusions

Contact Info

Product

Resources

About