A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines

Sajjad, Muhammad; Ullah, Fath U Min; Ullah, Mohib; Christodoulou, Georgia; Cheikh, Faouzi Alaya; Hijji, Mohammad; Muhammad, Khan; Rodrigues, Joel J. P. C.

doi:10.1016/j.aej.2023.01.017

Cited by 53 publications

(13 citation statements)

References 162 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gray & Wegner, 2012;H. M. Gray et al, 2007;Huebner, 2010;Kramer et al, 2018;Sajjad et al, 2023;Searle, 1980), trust in AI management should be lower than trust in human management. Second, emotional experience is seen as integral for a genuine desire to help others (Ashforth & Humphrey, 1993;Barasch et al, 2014;Grandey, 2003;Gross, 1998;van Kleef, 2016), so the lack of an ability to experience emotions should reduce the perceived benevolence of AI management compared to human management.…”

Section: Ai Management and Trustmentioning

confidence: 99%

“…We did not directly measure the capability of AI management to experience emotions in our studies. Although prior work has shown that AI is unable to experience emotions and that individuals recognize this limitation (Bigman & Gray, 2018;Kajale et al, 2023;Ng et al, 2023;Sajjad et al, 2023), researchers have begun to explore factors, such as anthropomorphism, that might increase the perceived emotional experience of AI. Although some studies suggest that making AI managers more humanlike might increase trust, others suggest that attempting to make them too humanlike can lead to backlash (Mori et al, 2012;Schroeder & Epley, 2016;Schroeder & Schroeder, 2018).…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

See 1 more Smart Citation

How perceived lack of benevolence harms trust of artificial intelligence management.

Li,

Bitterly

2024

Journal of Applied Psychology

View full text Add to dashboard Cite

As organizations continue to supplement and replace human management with artificial intelligence (AI), it is essential that we understand the factors that influence employees' trust in AI management. Across one preregistered field study, where we survey 400 delivery riders in Mainland China, and three preregistered experiments (total N = 2,350), we find that AI management is perceived as less benevolent than human management. Given that benevolence is an important antecedent of trust in leaders, this perception has a negative effect on trust in AI management, even when controlling for perceived ability and integrity. Employees prefer human management to AI management in high empathy demand contexts, where individuals seek management that can empathize and experience the emotions that they are feeling, as opposed to low empathy demand contexts. These findings deepen our understanding of trust and provide important theoretical and practical insights on the implementation and adoption of AI management.

show abstract

Section: Ai Management and Trustmentioning

confidence: 99%

Section: Limitations and Future Directionsmentioning

confidence: 99%

How perceived lack of benevolence harms trust of artificial intelligence management.

Li,

Bitterly

2024

Journal of Applied Psychology

View full text Add to dashboard Cite

show abstract

“…Learned features are directly extracted from raw data through deep neural networks (DNNs) comprising multiple layers that hierarchically learn more complex, representative spatial–temporal features than the previous layers. Accumulating research evidence suggests that DNN models consistently surpass most of the shallow learning models based on handcrafted features by a large margin [ 28 ]. Compared to shallow learning models, cross-domain experiments reveal that DNN models such as convolutional neural networks (CNNs) and vision transformers (ViTs) achieve superior generalizability and accuracy for emotion recognition [ 29 , 30 , 31 , 32 ] and AU detection [ 33 , 34 , 35 ] on unseen datasets with different demographics, camera views, and emotion-eliciting contexts.…”

Section: Analyzing Naturalistic Facial Expressions With Deep Learningmentioning

confidence: 99%

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

Bian,

Küster,

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.

show abstract

“…Multimodal emotion recognition [ 4 , 5 ] has been developed to address this restriction. The goal of multimodal emotion recognition is to enhance the reliability of emotion identification systems by including data from many modalities, such as facial expressions [ 6 , 7 ], spoken words [ 8 , 9 , 10 ], and text [ 11 , 12 ]. A person’s emotional state can be captured more accurately by combining many senses.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features

Dilnoza

Abdusalomov

Kutlimuratov

et al. 2023

Sensors

View full text Add to dashboard Cite

Methods for detecting emotions that employ many modalities at the same time have been found to be more accurate and resilient than those that rely on a single sense. This is due to the fact that sentiments may be conveyed in a wide range of modalities, each of which offers a different and complementary window into the thoughts and emotions of the speaker. In this way, a more complete picture of a person’s emotional state may emerge through the fusion and analysis of data from several modalities. The research suggests a new attention-based approach to multimodal emotion recognition. This technique integrates facial and speech features that have been extracted by independent encoders in order to pick the aspects that are the most informative. It increases the system’s accuracy by processing speech and facial features of various sizes and focuses on the most useful bits of input. A more comprehensive representation of facial expressions is extracted by the use of both low- and high-level facial features. These modalities are combined using a fusion network to create a multimodal feature vector which is then fed to a classification layer for emotion recognition. The developed system is evaluated on two datasets, IEMOCAP and CMU-MOSEI, and shows superior performance compared to existing models, achieving a weighted accuracy WA of 74.6% and an F1 score of 66.1% on the IEMOCAP dataset and a WA of 80.7% and F1 score of 73.7% on the CMU-MOSEI dataset.

show abstract

A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines

Cited by 53 publications

References 162 publications

How perceived lack of benevolence harms trust of artificial intelligence management.

How perceived lack of benevolence harms trust of artificial intelligence management.

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

Multimodal Emotion Detection via Attention-Based Fusion of Extracted Facial and Speech Features

Contact Info

Product

Resources

About