Vision Transformer With Attentive Pooling for Robust Facial Expression Recognition

Xue, Fanglei; Wang, Qiangchang; Tan, Zichang; Ma, Zhong‐Qi; Guo, Guodong

doi:10.1109/taffc.2022.3226473

Cited by 57 publications

(9 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…9. VTFF [28], TransFER [24], Facial Chirality [45], APViT [26], POSTER [25], POSTER++ [27], and ARBEx [47] are transformer-based architecture, whereas RAN [5], SCAN-CCI [9], ARM [44], EAC [43], and DDAMFN [46] are CNN- Methods # Params RAF-DB FERPlus VTFF [28] 80.1M -88.81 RAN [5] 11.2M 86.90 89.16 VTFF [28] 51.8M 88.14 -SCAN-CCI [9] 70M 89.02 89.42 EAC [43] 11.2M 89.99 89.64 ARM [44] 11.2M 90.42 -TransFER [24] 65.2M 90.91 90.83 Facial Chirality [45] 46.2M 91.20 -DDAMFN [46] 4.11M 91.35 90.74 APViT [26] 65.2M 91.98 90.86 POSTER [25] 71.8M 92.05 91.62 POSTER++ [27] 43 based architecture.…”

Section: Methodsmentioning

confidence: 99%

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

Ngwe,

Lim,

Lee

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is a machine learning problem that deals with recognizing human facial expressions. While existing work has achieved performance improvements in recent years, FER in the wild and under challenging conditions remains a challenge. In this paper, a lightweight patch and attention network based on MobileNetV1, referred to as PAtt-Lite, is proposed to improve FER performance under challenging conditions. A truncated ImageNet-pre-trained MobileNetV1 is utilized as the backbone feature extractor of the proposed method. In place of the truncated layers is a patch extraction block that is proposed for extracting significant local facial features to enhance the representation from MobileNetV1, especially under challenging conditions. An attention classifier is also proposed to improve the learning of these patched feature maps from the extremely lightweight feature extractor. The experimental results on public benchmark databases proved the effectiveness of the proposed method. PAtt-Lite achieved state-of-the-art results on CK+, RAF-DB, FER2013, FERPlus, and the challenging conditions subsets for RAF-DB and FERPlus.

show abstract

Section: Methodsmentioning

confidence: 99%

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

Ngwe,

Lim,

Lee

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…1, models with more parameters are not always better. APViT [32] is a recently proposed state-of-the-art method that combines both CNN and ViT for feature extraction. It boosts IR-50 from 30.78 to 35.48.…”

Section: Methodsmentioning

confidence: 99%

Exploring Expression-related Self-supervised Learning for Affective Behaviour Analysis

Xue¹,

Yang²

2023

Preprint

View full text Add to dashboard Cite

This paper explores an expression-related selfsupervised learning (SSL) method (ContraWarping) to perform expression classification in the 5th Affective Behavior Analysis in-the-wild (ABAW) competition. Affective datasets are expensive to annotate, and SSL methods could learn from large-scale unlabeled data, which is more suitable for this task. By evaluating on the Aff-Wild2 dataset, we demonstrate that Con-traWarping outperforms most existing supervised methods and shows great application potential in the affective analysis area. Codes will be released on: https : //github.com/youqingxiaozhua/ABAW5.

show abstract

“…A comparison method was selected among SOTA FER studies specialised for wild FER with 1) a patch-based attention CNN mechanism (pACNN) [23], 2) adversarial graph representation (AGR) [55], 3) region attention network (RAN) [32], 4) feature decomposition and reconstruction learning (FDRL) [28], 5) EfficientFace [56], 6) deep attentive centre loss (DACL) [20], 7) latent distribution mining and pairwise uncertainty estimation (DMUE) [57], 8) relative uncertainty learning (RUL) [58], 9) FER with visual transformers with feature fusion (VTFF) [59], 10) squeeze vision transformer (Squeeze-ViT) [35], 11) FER through the meta (Face2Exp) [29], and 12) vision transformer with attention pooling (APViT) [60]. As shown in Table 1, the proposed method combining the face graph and a GCN showed the best performance in all three datasets, except AffectNet-8.…”

Section: A Performance Comparison With State-of-art Approachesmentioning

confidence: 99%

“…However, because AffectNet-8 used pre-labelled landmarks to train the model, there was a slight performance degradation compared to other datasets. APViT [60] is a method that combines CNNs and vision transformers, and showed good performance in both AffectNet-8 and RAF-DB because it was pre-trained with different datasets. However, when only scratch learning was applied without pre-training like our method, the performance was very poor compared to the proposed method.…”

Section: A Performance Comparison With State-of-art Approachesmentioning

confidence: 99%

Facial Expression Recognition in the Wild Using Face Graph and Attention

Kim

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Facial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expressions, and we propose an algorithm for recognizing facial expressions using a graph convolutional network (GCN). We first generated an attention map that can highlight action units to extract important facial expression features from faces in the wild. After feature extraction, a face graph is constructed by combining the attention map with face patches, and changes in expression in the wild are recognized using a GCN. Through comparative experiments conducted using both lab-controlled and wild datasets, we prove that the proposed method is the most suitable FER approach for use with image datasets captured in the wild and those under well-controlled indoor conditions.

show abstract

Vision Transformer With Attentive Pooling for Robust Facial Expression Recognition

Cited by 57 publications

References 60 publications

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

Exploring Expression-related Self-supervised Learning for Affective Behaviour Analysis

Facial Expression Recognition in the Wild Using Face Graph and Attention

Contact Info

Product

Resources

About