Visual Saliency Maps Can Apply to Facial Expression Recognition

Qin, Zhenyue; Wu, Jie

doi:10.48550/arxiv.1811.04544

Cited by 6 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, first cut the upper left corner, lower left corner, upper right corner, lower right corner and center of the image and then perform horizontal flipping operations on them, respectively, expanding the number of datasets by a factor of 10. That is, for an image, the 10 images obtained after processing are sent to the model, and then the probability values obtained from each image are averaged to obtain 7 average probability values, and the expression class corresponding to the maximum probability value is taken as the final output expression [48].…”

Section: Data Pre-processingmentioning

confidence: 99%

Facial expression recognition based on regional adaptive correlation

Meng

Yuan

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

To address the problem that the features extracted by CNN‐based facial expression recognition (FER) do not consider structural information, a region adaptive correlation deep network (RACN) is proposed. The network consists of two branches. In one branch, the features obtained by applying CNN to facial sub‐blocks are used as the input of the proposed second‐order region correlation network (SRCN), which obtains structural features by adaptively learning the correlation of facial regions. Furthermore, they are fused with the parallel branch‐extracted global features to obtain a comprehensive high‐semantic feature representation. Finally, weights are assigned to the two features through the channel attention mechanism for more accurate expression classification. Experimental results show that our method can effectively extract expression features in an end‐to‐end manner, improve the accuracy of FER, and achieve competitive performance without relying on any a priori knowledge. And the region‐adaptive correlation feature extraction branch RACN can be applied to other deep learning networks to extract discriminative structural‐adaptive features. To the best of our knowledge, our work is the first to enrich the feature representation for end‐to‐end static FER by adaptively obtaining more discriminative regional adaptive correlation feature vectors via the autocorrelation matrix combined with CNN compared to the existing literature.

show abstract

Section: Data Pre-processingmentioning

confidence: 99%

Facial expression recognition based on regional adaptive correlation

Meng

Yuan

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…Mollahosseini et al [21] applied a convolutional neural network to FER. Qin and Wu [22] utilized Resnet [23] to achieve good recognition performance on multiple datasets. Ding et al [24] proposed to train an expression recognition network called the FaceNet2E xpNet based on images.…”

Section: Spatial-based Approachesmentioning

confidence: 99%

Transformer-based Siamese and Triplet Networks for Facial Expression Intensity Estimation

Sabri¹

2023

IJAE

View full text Add to dashboard Cite

Recognizing facial expressions and estimating their corresponding action units' intensities have achieved many milestones. However, such estimating is still challenging due to subtle action units' variations during emotional arousal. The latest approaches are confined to the probabilistic models' characteristics that model action units' relationships. Considering ordinal relationships across an emotional transition sequence, we propose two metric learning approaches with self-attention-based triplet and Siamese networks to estimate emotional intensities. Our emotion expert branches use shifted-window SWIN-transformer which restricts self-attention computation to adjacent windows while also allowing for cross-window connection. This offers flexible modeling at various scales of action units with high performance. We evaluated our network's spatial and time-based feature localization on CK+, KDEF-dyn, AFEW, SAMM, and CASME-II datasets. They outperform deep learning state-of-the-art methods in micro-expression detection on the latter two datasets with 2.4% and 2.6% UAR respectively. Ablation studies highlight the strength of our design with a thorough analysis.

show abstract

“…The experimental results are shown in Figure 7. It turns out Methods Years Accuracy(%) Conv+Inception [30] 2016 66.40 SMFER [31] 2018 65.12 GoogleNet [32] 2018 65.20 VGG+SVM [33] 2019 66.31 SAP [34] 2019 71.08 E-FCNN [28] 2021 66.17 DeepEmotion [35] 2021 uses transfer learning technique to overcome the shortage of training samples. For FER2013, the second-highest recognition accuracy of the test set is SAP [34], which is a sample awareness-based expression recognition method, in which a Bayesian classifier is used to select the most appropriate classifier from a set of candidate classifiers for the current test sample, and then the classifier is used to perform expression recognition on the current sample.…”

Section: E Visualization Analysismentioning

confidence: 99%

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

Yang

Kuang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The sample imbalance of expression datasets always leads to poor recognition results for minority classes. To solve this problem, we propose a facial expression recognition network, called Residual Attentive Sharing Network (RASN). There is a fact that different expressions have affinity features, which makes it possible for the minority classes to benefit from the majority classes in the expression feature extraction process, from which we propose a sharing affinity features module to compensate for the inadequate feature learning of minority classes by sharing affinity features. In addition, an affinity features attention module is added to highlight expression-related affinity features and suppress expression-unrelated ones for enhancing the role of sharing affinity features. Experiments on the CK+, RAF-DB, and FER2013 datasets validate the robustness of our method to sample imbalance. The validation accuracies of our method are 96.97% on CK+, 71.44% on FER2013, and 90.91% on RAF-DB, respectively, which exceed current state-of-the-art methods.

show abstract

Visual Saliency Maps Can Apply to Facial Expression Recognition

Cited by 6 publications

References 27 publications

Facial expression recognition based on regional adaptive correlation

Facial expression recognition based on regional adaptive correlation

Transformer-based Siamese and Triplet Networks for Facial Expression Intensity Estimation

RASN: Using Attention and Sharing Affinity Features to Address Sample Imbalance in Facial Expression Recognition

Contact Info

Product

Resources

About