Graph-Based Facial Affect Analysis: A Review

Abstract

As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This paper provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

Advances in Facial Expression Recognition: A Survey of Methods, Benchmarks, Models, and Datasets

“…Graph-based methodologies have emerged as a compelling framework for capturing the complex relationships among various facial features and expressions, thereby enabling a nuanced and comprehensive affective analysis [271,272]. These techniques have exhibited marked improvements in both the performance and generalizability of facial expression recognition models when compared to traditional methods that predominantly focus on appearance and geometric features.…”

Emotion-Recognition Algorithm Based on Weight-Adaptive Thought of Audio and Video

“…The study implements emotion recognition and intensity estimation for each recognized emotion. Yang Liu et al conduct the first investigation of the graph-based FAA method [18]. The results of the team's findings can serve as a reference for future research in this area and summarize the performance comparison of state-of-the-art graph-based FAA methods, discussing the challenges and potential directions for future development.…”

Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion

“…Recently, there have been some other surveys and reviews on artificial EQ (AEI), such as facial expression recognition (FER) [ 3 ], [ 4 ], [ 5 ], [ 6 ], microexpression recognition (MER) [ 7 ], [ 8 ], [ 9 ], textual sentiment classification [ 10 ], [ 11 ], [ 12 ], music and speech emotion recognition [ 13 ], [ 14 ], [ 15 ], affective image content analysis [ 16 ], emotional body gesture recognition [ 17 ], bodily expressed emotion recognition [ 18 ], emotion recognition from physiological signals [ 19 ], [ 20 ], multimodal emotion recognition [ 21 ], [ 22 ], and affective theory use [ 23 ]. These articles mainly focus on emotion and sentiment analysis for a specific modality from the perspective of machine learning and pattern recognition or focus on the psychological emotion theories.…”