Learning a Facial Expression Embedding Disentangled from Identity

Zhang, Wei; Ji, Xianpeng; Chen, Keyu; Ding, Yu; Fan, Changjie

doi:10.1109/cvpr46437.2021.00669

Cited by 61 publications

(25 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy(%) Average Accuracy(%) DLP-CNN [43] 84.22 74.20 FSN [51] 81.10 72.46 paCNN [45] 83.27 -gaCNN [47] 85.07 -IPA2LT [46] 86.77 -ALT [52] 84.50 76.50 separate loss [34] 86.38 77.25 WS-LGAN [53] 85.07 -RAN [48] 86.90 -DLN [18] 86.4 -SCN [49] 87.03 -Ad-Corre 86.96 79.01 Disgust and Anger (about 13%), while most confusion, about 18%, occurred between Sad and Neutral. In addition, we report the precision, recall, and f1-score of our proposed Ad-Corre Loss on AffectNet [44], RAF-DB [43] and FER-2013 [42] datsets in Tables 5, 6, 7 respectively.…”

Section: Methodsmentioning

confidence: 99%

“…Dharanya et al [16] proposed Auxiliary Classifier Generative Adversarial Network (AC-GAN) based model which regenerates the basic facial emotions from an input face image and then classifies them. Zhang et al [18] proposed a weakly supervised local-global attention network which is designed to extract and combine the local and the global features from input facial images. Also, their proposed architecture is designed to use the attention mechanism to deal with part location and feature fusion problems.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

View full text Add to dashboard Cite

Automated Facial Expression Recognition (FER) in the wild using deep neural networks is still challenging due to intra-class variations and inter-class similarities in facial images. Deep Metric Learning (DML) is among the widely used methods to deal with these issues by improving the discriminative power of the learned embedded features. This paper proposes an Adaptive Correlation (Ad-Corre) Loss to guide the network towards generating embedded feature vectors with high correlation for within-class samples and less correlation for between-class samples. Ad-Corre consists of 3 components called Feature Discriminator, Mean Discriminator, and Embedding Discriminator. We design the Feature Discriminator component to guide the network to create the embedded feature vectors to be highly correlated if they belong to a similar class, and less correlated if they belong to different classes. In addition, the Mean Discriminator component leads the network to make the mean embedded feature vectors of different classes to be less similar to each other. We use Xception network as the backbone of our model, and contrary to previous work, we propose an embedding feature space that contains k feature vectors. Then, the Embedding Discriminator component penalizes the network to generate the embedded feature vectors, which are dissimilar. We trained our model using the combination of our proposed loss functions called Ad-Corre Loss jointly with the cross-entropy loss. We achieved a very promising recognition accuracy on AffectNet, RAF-DB, and FER-2013. Our extensive experiments and ablation study indicate the power of our method to cope well with challenging FER tasks in the wild.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Due to the limited subjects and unbalanced annotations of existed affective datasets, it is a challenging issue to prevent the emotion recognition model from overfitting on the disturbing factors, like background or random noise. To tackle this problem, we adopt a prior facial expression embedding model [13], which can capture the detailed expression similarities across different people, into our framework. The expression embedding brings at least two advantages.…”

Section: Overviewmentioning

confidence: 99%

“…First, by training on even larger facial image datasets with the identity invariant constraint, the embedding itself is independent to the identity attributes and therefore can improve the network's generalizability to unseen subjects. Second, the expression embedding model [13] is targeted for discriminating the minor expression similarities within triplet training data. It provides a nice initialization for our latter emotion recognition tasks.…”

Section: Overviewmentioning

confidence: 99%

“…Despite the traditional facial expression recognition (FER) arXiv:2107.03708v1 [cs.CV] 8 Jul 2021 models have regressed continuous expression distributions for discrete classification, they can hardly encode the finegrained expression features. In this work, we adopt the triplet-based expression embedding [13] model as the backbone of the entire framework. Since the expression embedding is trained to distinguish minor expression similarities between different subjects, it can provide powerful expression-related priors to the high-level emotion recognition task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prior Aided Streaming Network for Multi-task Affective Analysis

Zhang¹,

Guo²,

Chen³

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

Self Cite

View full text Add to dashboard Cite

Automatic affective recognition has been an important research topic in human computer interaction (HCI) area. With recent development of deep learning techniques and large scale in-the-wild annotated datasets, the facial emotion analysis is now aimed at challenges in the real world settings. In this paper, we introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, including Categorical Emotions (CE), Action Units (AU), and Valence Arousal (VA), we propose a multi-task streaming network by a heuristic that the three representations are intrinsically associated with each other. Besides, we leverage an advanced facial expression embedding as prior knowledge, which is capable of capturing identity-invariant expression features while preserving the expression similarities, to aid the down-streaming recognition tasks. The extensive quantitative evaluations as well as ablation studies on the Aff-Wild2 dataset prove the effectiveness of our proposed prior aided streaming network approach.

show abstract

Facial Expression Recognition In-the-Wild with Deep Pre-trained Models

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning a Facial Expression Embedding Disentangled from Identity

Cited by 61 publications

References 28 publications

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Prior Aided Streaming Network for Multi-task Affective Analysis

Facial Expression Recognition In-the-Wild with Deep Pre-trained Models

Contact Info

Product

Resources

About