Facial Expression Recognition based on Multi-head Cross Attention Network

Jeong, Jae-Yeop; Hong, Yeong-Gi; Kim, Daun; Jung, Yuchul; Jeong, Jin-Woo

doi:10.48550/arxiv.2203.13235

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the 3 th ABAW Competition 2022, Zhang et.al [30] introduced a transformerbased fusion module that extracted the visual features and the dynamic spatial-temporal features and ranked the 1 st place. Jeong et.al [7] adopted an extended DAN model to address the facial expression.…”

Section: Facial Expression Classificationmentioning

confidence: 99%

Spatial-temporal Transformer for Affective Behavior Analysis

Zou¹,

Wang²,

Wen³

et al. 2023

Preprint

View full text Add to dashboard Cite

The in-the-wild affective behavior analysis has been an important study. In this paper, we submit our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes V-A Estimation, Facial Expression Classification and AU Detection Sub-challenges. We propose a Transformer Encoder with Multi-Head Attention framework to learn the distribution of both the spatial and temporal features. Besides, there are virious effective data augmentation strategies employed to alleviate the problems of sample imbalance during model training. The results fully demonstrate the effectiveness of our proposed model based on the Aff-Wild2 dataset.

show abstract

Section: Facial Expression Classificationmentioning

confidence: 99%

Spatial-temporal Transformer for Affective Behavior Analysis

Zou¹,

Wang²,

Wen³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Zhang et al [37] proposed a transformer-based fusion module to fuse multi-modality features from audio, image, and word information. Jeong et al [11] extended the DAN model and achieved 2nd in ABAW3. Xue et al [30] utilized a coarseto-fine cascade network with a temporal smoothing strategy and ranked 3rd in ABAW3.…”

Section: Related Workmentioning

confidence: 99%

Exploring Expression-related Self-supervised Learning for Affective Behaviour Analysis

Xue¹,

Yang²

2023

Preprint

View full text Add to dashboard Cite

This paper explores an expression-related selfsupervised learning (SSL) method (ContraWarping) to perform expression classification in the 5th Affective Behavior Analysis in-the-wild (ABAW) competition. Affective datasets are expensive to annotate, and SSL methods could learn from large-scale unlabeled data, which is more suitable for this task. By evaluating on the Aff-Wild2 dataset, we demonstrate that Con-traWarping outperforms most existing supervised methods and shows great application potential in the affective analysis area. Codes will be released on: https : //github.com/youqingxiaozhua/ABAW5.

show abstract

“…Therefore, it is worth exploring how to use static images for a more generalized usage scenario. Jeong et al [9] use an affinity loss approach, which uses affinity loss to train a feature extractor for images. In addition, they propose a multi-head attention network in a coordinated manner to extract diverse attention for EXPR.…”

Section: Related Workmentioning

confidence: 99%

Affective Behaviour Analysis Using Pretrained Model with Facial Priori

Li¹,

Sun²,

Zhaori³

et al. 2022

Preprint

View full text Add to dashboard Cite

Affective behaviour analysis has aroused researchers' attention due to its broad applications. However, it is labor exhaustive to obtain accurate annotations for massive face images. Thus, we propose to utilize the prior facial information via Masked Auto-Encoder (MAE) pretrained on unlabeled face images. Furthermore, we combine MAE pretrained Vision Transformer (ViT) and AffectNet pretrained CNN to perform multi-task emotion recognition. We notice that expression and action unit (AU) scores are pure and intact features for valence-arousal (VA) regression. As a result, we utilize AffectNet pretrained CNN to extract expression scores concatenating with expression and AU scores from ViT to obtain the final VA features. Moreover, we also propose a co-training framework with two parallel MAE pretrained ViT for expression recognition tasks. In order to make the two views independent, we random mask most patches during the training process. Then, JS divergence is performed to make the predictions of the two views as consistent as possible. The results on ABAW4 show that our methods are effective.

show abstract

Facial Expression Recognition based on Multi-head Cross Attention Network

Cited by 3 publications

References 9 publications

Spatial-temporal Transformer for Affective Behavior Analysis

Spatial-temporal Transformer for Affective Behavior Analysis

Exploring Expression-related Self-supervised Learning for Affective Behaviour Analysis

Affective Behaviour Analysis Using Pretrained Model with Facial Priori

Contact Info

Product

Resources

About