Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild

Abstract

Compared with the image-based static facial expression recognition (SFER) task, the dynamic facial expression recognition (DFER) task based on video sequences is closer to the natural expression recognition scene. However, DFER is often more challenging. One of the main reasons is that video sequences often contain frames with different expression intensities, especially for the facial expressions in the real-world scenarios, while the images in SFER frequently present uniform and high expression intensities. Nevertheless, if the expressions with different intensities are treated equally, the features learned by the networks will have large intra-class and small inter-class differences, which are harmful to DFER. To tackle this problem, we propose the global convolution-attention block (GCA) to rescale the channels of the feature maps. In addition, we introduce the intensity-aware loss (IAL) in the training process to help the network distinguish the samples with relatively low expression intensities. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39k) indicate that our method outperforms the state-of-the-art DFER approaches. The source code will be available at https://github.com/muse1998/IAL-for-Facial-Expression-Recognition.

SlowR50-SA: A Self-Attention Enhanced Dynamic Facial Expression Recognition Model for Tactile Internet Applications

“…Its goal is to classify a facial video clip, rather than a still image, into one of the basic emotions. The field of DFER has attracted considerable attention from researchers [16][17][18][19][20][21][22]. These studies share a common goal of addressing challenges within environmental scenarios, such as occlusion, pose variation, and noisy frames.…”

“…This method incorporates emotion-driven loss functions to enhance recognition accuracy, but it may lack robustness in handling diverse environmental scenarios. Li et al (2023) [22] contributed to intensity-adaptive loss for dynamic facial expression recognition by integrating a global attentional bias (GCA) block and intensity-adaptive loss (IAL) to handle different expression intensities. While effective in addressing intensity variations, this approach may require additional computational overhead.…”

“…Wang et al (2023) [20] reimagined the learning paradigm for DFER by treating it as a weakly supervised problem and introduced the multi-3D dynamic facial expression learning (M3DFEL) framework with multi-instance learning (MIL). Additionally, variations in loss functions are investigated with Li et al ( 2023) [22], introducing Intensity-Aware Loss to distinguish samples with low expression intensity. While the Intensity-Aware Loss effectively handles expression intensity variations, it may introduce additional computational overhead during training, potentially limiting scalability to larger datasets.…”

“…This method emphasizes the efficacy of proposed loss functions and fusion parameters. Addressing the nuances of expression intensity, the Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild method [22] employs a GCA Block, Dynamic-Static Fusion Module, and Temporal Transformer for feature extraction. Trained on PyTorch-GPU and Tesla V100 GPUs, it achieves performance on dynamic facial expression recognition tasks.…”

“…In this study, UAR and WAR are employed as primary evaluation metrics, aligning with established practices in the field of dynamic facial expression recognition. These metrics are widely used in previous studies for their effectiveness in evaluating model performance across various domains, including facial expression recognition [17][18][19][20]22,27]. UAR, computed as the average recall across all classes, provides an unbiased assessment of the model's ability to accurately classify facial expressions without favoring any specific class.…”