Facial Expression Recognition: A Review of Trends and Techniques

Ekundayo, Olufisayo; Viriri, Serestina

doi:10.1109/access.2021.3113464

Cited by 42 publications

(10 citation statements)

References 203 publications

(195 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…F ACIAL expressions are an incredibly complex and dynamic communication tool capable of expressing a wide range of emotions [1,2]. Researchers have long been interested in accurately estimate emotions from facial expressions for decades, as it has implications in various fields such as neuroscience, human-computer interaction, marketing, and psychomotor learning [3,4,5,6,7,8].…”

Section: Introductionmentioning

confidence: 99%

Exploring the Impact of Partial Occlusion on Emotion Classification From Facial Expressions: A Comparative Study of XR Headsets and Face Masks

Casas-Ortiz,

Echeverria,

Jimenez-Tellez

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This study provides of a comparative analysis of emotion estimation from facial expressions under partial occlusion caused by face masks and extended reality (XR) headsets. Unlike previous studies that have independently explored these two scenarios, this research compares and analyzes the statistical differences between them. In order to achieve this, the RAF-DB dataset has been used as a non-occluded baseline to construct two new datasets: i) a dataset formed by faces partially occluded by face masks, and ii) a dataset formed by faces partially occluded by XR headsets. To evaluate the impact of occlusion in emotion estimation, three deep learning models have been finetuned using transfer learning, and results from a random classifier have been used as a baseline. Seven different metrics were obtained per dataset, and a 2-way ANOVA test was performed on each metric. As expected, significant statistical differences are observed between the non-occluded faces (acc. 0.8780) and the faces partially occluded by face masks (acc. 0.7520) and XR headsets (acc. 0.7400) on all metrics. Notably, the comparison between the two partially occluded datasets revealed significant statistical differences in the metrics f1-score (macro), precision (macro) and recall (macro), which we attribute to different types of occlusion affecting different parts of the face that are key to some emotions. This research contributes to advancing emotion recognition systems by highlighting their robustness and effectiveness even in partial occlusion settings, and showing a full comparative analysis between two common types of occlusion.

show abstract

Section: Introductionmentioning

confidence: 99%

Exploring the Impact of Partial Occlusion on Emotion Classification From Facial Expressions: A Comparative Study of XR Headsets and Face Masks

Casas-Ortiz,

Echeverria,

Jimenez-Tellez

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…By contrast, recent AFERSs based on deep-learning (DL) neural networks are end-to-end systems that automatically learn to extract facial features for accurate expression classification and hence reduce the need for image preprocessing and feature extraction ( O’Mahony et al, 2020 ; Khan, 2022 ). While DL-based AFERSs require a large number of face images as training data, they outperform traditional approaches and have become state-of-the-art ( Li and Deng, 2020 ; Ekundayo and Viriri, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

The cross-race effect in automatic facial expression recognition violates measurement invariance

Li,

Yeh,

Huang

2023

Front. Psychol.

View full text Add to dashboard Cite

Emotion has been a subject undergoing intensive research in psychology and cognitive neuroscience over several decades. Recently, more and more studies of emotion have adopted automatic rather than manual methods of facial emotion recognition to analyze images or videos of human faces. Compared to manual methods, these computer-vision-based, automatic methods can help objectively and rapidly analyze a large amount of data. These automatic methods have also been validated and believed to be accurate in their judgments. However, these automatic methods often rely on statistical learning models (e.g., deep neural networks), which are intrinsically inductive and thus suffer from problems of induction. Specifically, the models that were trained primarily on Western faces may not generalize well to accurately judge Eastern faces, which can then jeopardize the measurement invariance of emotions in cross-cultural studies. To demonstrate such a possibility, the present study carries out a cross-racial validation of two popular facial emotion recognition systems—FaceReader and DeepFace—using two Western and two Eastern face datasets. Although both systems could achieve overall high accuracies in the judgments of emotion category on the Western datasets, they performed relatively poorly on the Eastern datasets, especially in recognition of negative emotions. While these results caution the use of these automatic methods of emotion recognition on non-Western faces, the results also suggest that the measurements of happiness outputted by these automatic methods are accurate and invariant across races and hence can still be utilized for cross-cultural studies of positive psychology.

show abstract

“…Convolutional neural networks (CNNs) are a significant subset of artificial intelligence, as mentioned in the previous article [11], [12]. The performance accuracy will be higher with visual detection [13] of the driver's condition using CNN as input data in real-time for the forward collision warning (FCW) system. The forward-collision alert system can provide an early warning before collisions with an object in front of the vehicle [14].…”

Section: Introductionmentioning

confidence: 99%

Artificial intelligence system for driver distraction by stacked deep learning classification

Qibtiah

Zin²,

Hassan³

2023

Bulletin EEI

View full text Add to dashboard Cite

Increasing efforts in the transportation system have recently improved driver safety and reduced crash rates. Lack of attention and fatigue directly affect the driver's consciousness. Driver distraction is an essential driver-specific factor in the practical applicability of forward collision warning (FCW). However, there are still too many false alarms generated by the existing FCW system to be mitigated. This paper proposes facial detection to identify features and test anomalies' prediction against drivers using stacked convolutional neural network (CNN) layers. The proposed model used overlapping HAAR and stacked CNN features to identify classifications of eye areas, such as open or closed. In addition to the sliding query window's overall intensity information. The conventional HAAR function, which elevates the brightness of nearby regions, is still preferable. This method considers current intelligent transportation system-based solutions to minimize distraction effects by continuously comparing with flexible thresholds. The experimental results are analyzed from accurate driving datasets. At 456 iterations, the results acquired over 80% accuracy, while loss is near zero. The implication of driver's risk tolerance is further explored in this manner. Several risks are connected to driving any type of transportation system.

show abstract

Facial Expression Recognition: A Review of Trends and Techniques

Cited by 42 publications

References 203 publications

Exploring the Impact of Partial Occlusion on Emotion Classification From Facial Expressions: A Comparative Study of XR Headsets and Face Masks

Exploring the Impact of Partial Occlusion on Emotion Classification From Facial Expressions: A Comparative Study of XR Headsets and Face Masks

The cross-race effect in automatic facial expression recognition violates measurement invariance

Artificial intelligence system for driver distraction by stacked deep learning classification

Contact Info

Product

Resources

About