Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition

Abstract

We present a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations. Firstly, multiple classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions exhibit themselves through multiple facial regions simultaneously, and the recognition requires a holistic approach by encoding high-order interactions among local features. To address these issues, we propose our DAN with three key components: Feature Clustering Network (FCN), Multi-head cross Attention Network (MAN), and Attention Fusion Network (AFN). The FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, the MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, the AFN distracts these attentions to multiple locations before fusing the attention maps to a comprehensive one. Extensive experiments on three public datasets (including Affect-Net, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. Code will be made available at: https://github.com/yaoing/DAN.

Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition

“…As a result, for both gender and race groups, CL models achieve high fairness scores by balancing performance across the domain splits while also offering competitive accuracy scores, with the SI model performing the best in terms of fairness (see Table 11). Yet, NR achieves a competitive average accuracy score (across domain groups) of 0.771 for gender and 0.807 for race evaluations, highest amongst the compared methods, with the benchmark evaluations on RAF-DB at 0.853 [58]. Selective updates of model parameters to mitigate forgetting allows CL models to maintain high accuracy scores across the different gender and race attributes.…”

Facial Expression Recognition based on Multi-head Cross Attention Network

“…Afterwards, a multihead attention network consisting of a combination of a spatial attention unit and a channel attention unit takes the features and outputs an attention map. Finally, attention fusion network merges attention maps to be learned in an orchestrated fashion [10]. Second, after applying the process mentioned above, the final feature information is fed to fully connected layer and batch normalization layer.…”

“…As evaluation metrics, mean Concordance Correlation Coefficient (CCC) of valence and arousal and the average F1 Score across all 8 categories were used for the VA estimation challenge and the expression classification challenge, respectively. In this paper, we propose an extended version of DAN model based on ResNet with attention mechanisms proposed by [10] to solve the challenges mentioned Figure 1: Overview of the architecture used in this study above , and present the preliminary results on the official validation set. More precise and detailed results can be updated and added through the subsequent submissions to the competition.…”

Blindfold Attention: Novel Mask Strategy for Facial Expression Recognition

“…Humans have a remarkable ability to understand the emotion or what people want to convey since we can attention on major expression features, such as, winking, mouth opening, sniffing, tilting. Accordingly, many studies focus on capturing and strengthen the local feature using attention mechanism [27,28,49] and retrieve promising results. Attention mechanism can be treated as a dynamic process which select adaptive weighting features according to the different input [11].…”