ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection &amp; Emotional Reaction Intensity Estimation Challenges

Kollias, Dimitrios; Tzirakis, Panagiotis; Baird, Alice; Cowen, Alan; Zafeiriou, Stefanos

doi:10.48550/arxiv.2303.01498

Cited by 9 publications

(14 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baseline [20] - Here, we employ Mean Square Error (MSE) loss to constrain the difference between the reconstructed patches and the original patches at the pixel-level. Suppose that M patches are masked at the beginning, the pre-training loss L pre is formulated as:…”

Section: Au1 Au2 Au4 Au6 Au7 Au10 Au12 Au15 Au23 Au24 Au25 Au26 Averagementioning

confidence: 99%

“…Dataset for AU detection: The AU Detection Challenge at 5th ABAW Competition [20] provides 541 video sequences from Aff-Wild2 dataset. Each frame of a video sequence in this dataset is manually or automatically annotated with labels of 12 AUs, namely AU1, AU2, AU4, AU6, AU7, AU10, AU12, AU15, AU23, AU24, AU25, and AU26.…”

Section: Datasetmentioning

confidence: 99%

“…The Action Unit Detection Challenge of the 5th ABAW Competition [20] is based on the Aff-Wild2 [16-19, 21-24, 55] database. Some of the AU detection approaches in the previous ABAW Competitions [16,17,24] fuse multimodal features including video and audio to provide multidimensional information to predict AUs' occurrence [13,14,50,58].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

Wang¹,

Song²,

Luo³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper presents our Facial Action Units (AUs) recognition submission to the fifth Affective Behavior Analysis in-the-wild Competition (ABAW). Our approach consists of three main modules: (i) a pre-trained facial representation encoder which produce a strong facial representation from each input face image in the input sequence; (ii) an AUspecific feature generator that specifically learns a set of AU features from each facial representation; and (iii) a spatiotemporal graph learning module that constructs a spatiotemporal graph representation. This graph representation describes AUs contained in all frames and predicts the occurrence of each AU based on both the modeled spatial information within the corresponding face and the learned temporal dynamics among frames. The experimental results show that our approach outperformed the baseline and the spatio-temporal graph representation learning allows our model to generate the best results among all ablated systems. Our model ranks at the 4th place in the AU recognition track at the 5th ABAW Competition.

show abstract

Section: Au1 Au2 Au4 Au6 Au7 Au10 Au12 Au15 Au23 Au24 Au25 Au26 Averagementioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

Wang¹,

Song²,

Luo³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…From a machine learning perspective, AU detection in the wild presents many technical challenges. Most notably, in-the-wild datasets such as Aff-Wild2 [12][13][14][15][16][17][18][19][20][21]32] collect data with huge variations in the cameras (resulting in blurred video frames), environments (illumination conditions), and subjects (large variance in expressions, scale, and head poses). Ertugrul et al [4,5] demonstrate that the deep-learning-based AU detectors have limited generalization abilities due to the aforementioned variations.…”

Section: Introductionmentioning

confidence: 99%

Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild

Yin¹,

Tran²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Facial action unit detection has emerged as an important task within facial expression analysis, aimed at detecting specific pre-defined, objective facial expressions, such as lip tightening and cheek raising. This paper presents our submission to the Affective Behavior Analysis in-the-wild (ABAW) 2023 Competition for AU detection. We propose a multi-modal method for facial action unit detection with visual, acoustic, and lexical features extracted from the large pre-trained models. To provide high-quality details for visual feature extraction, we apply super-resolution and face alignment to the training data. Our approach achieves the F1 score of 52.3% on the official validation set.

show abstract

“…al. [10][11][12][13][14][15][16][17][18][19]28] proposed Aff-Wild2 containing the above three representations in the wild. There are various challenges in this dataset, such as head poses, ages, sex, etc.…”

Section: Introductionmentioning

confidence: 99%

Spatial-temporal Transformer for Affective Behavior Analysis

Zou¹,

Wang²,

Wen³

et al. 2023

Preprint

View full text Add to dashboard Cite

The in-the-wild affective behavior analysis has been an important study. In this paper, we submit our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes V-A Estimation, Facial Expression Classification and AU Detection Sub-challenges. We propose a Transformer Encoder with Multi-Head Attention framework to learn the distribution of both the spatial and temporal features. Besides, there are virious effective data augmentation strategies employed to alleviate the problems of sample imbalance during model training. The results fully demonstrate the effectiveness of our proposed model based on the Aff-Wild2 dataset.

show abstract

ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges

Abstract: The fifth Affective Behavior Analysis in-the-wild (ABAW) Competition is part of the respective ABAW Workshop which will be held in conjunction with IEEE Computer Vision and

Cited by 9 publications

References 5 publications

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild

Spatial-temporal Transformer for Affective Behavior Analysis

Contact Info

Product

Resources

About