POSTER++: A simpler and stronger facial expression recognition network

Mao, Jiang‐Gao; Xu, Rui; Yin, Xiuli; Chang, Yuanqi; Huang, Aibin

doi:10.48550/arxiv.2301.12149

Cited by 11 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, researchers also have begun to introduce the Transformer or the ViT architecture for FER in recent years [23]- [29], motivated by their performance achieved across different tasks. Based on the results posted in existing work, the application of vision transformers in FER is proven to be useful with ViT+SE [23] posting the state-of-the-art performance of 99.80% mean accuracy across 10 folds on the CK+ database, POSTER++ [27] being the best-performing method on the RAF-DB database with 92.21% accuracy, and POSTER [25] being the second best-performing method on the FERPlus database by achieving 91.62% accuracy. However, this performance often comes with large architectures with significantly more parameters than CNN-based methods.…”

Section: B Vision Transformersmentioning

confidence: 97%

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

Ngwe,

Lim,

Lee

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is a machine learning problem that deals with recognizing human facial expressions. While existing work has achieved performance improvements in recent years, FER in the wild and under challenging conditions remains a challenge. In this paper, a lightweight patch and attention network based on MobileNetV1, referred to as PAtt-Lite, is proposed to improve FER performance under challenging conditions. A truncated ImageNet-pre-trained MobileNetV1 is utilized as the backbone feature extractor of the proposed method. In place of the truncated layers is a patch extraction block that is proposed for extracting significant local facial features to enhance the representation from MobileNetV1, especially under challenging conditions. An attention classifier is also proposed to improve the learning of these patched feature maps from the extremely lightweight feature extractor. The experimental results on public benchmark databases proved the effectiveness of the proposed method. PAtt-Lite achieved state-of-the-art results on CK+, RAF-DB, FER2013, FERPlus, and the challenging conditions subsets for RAF-DB and FERPlus.

show abstract

Section: B Vision Transformersmentioning

confidence: 97%

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

Ngwe,

Lim,

Lee

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The POSTER based network architecture is optimized and improved for the FER task, but the network structure of POSTER is not only complex but also leads to expensive computational costs, so in then based on POSTER, POSTER++ is proposed in order to alleviate the computational pressure. POSTER++ improves on POSTER in three ways: Cross Fusion, Dual Streaming and Multi-scale Feature Extraction [4].…”

Section: Postermentioning

confidence: 99%

A comparative study of deep convolutional neural network-based facial expression regression

Sun

2024

ACE

View full text Add to dashboard Cite

Facial expression is an important way for people to express their emotions. Recently, with the advancement of the computer vision, facial expression recognition has become a current research hotspot and has made significant progress, which can be utilized in the interaction between humans and computers, emotional computing and other computer vision fields, the evolution of artificial intelligence and deep learning has better promoted the research of facial expression recognition. Conventional approaches are largely based on machine learning, which leverages artificial way for feature extraction. The extracted facial expression features are interfered by human factors, so that the trained classifier cannot effectively interpret the expression information, which ultimately leads to insufficient model generalization ability and low recognition accuracy. However, deep learning-based facial expression recognition in real scenes is still affected by factors such as human pose, different degrees of facial occlusion, background environment and light interference, and the recognition accuracy still needs to be further improved. This paper focuses on representative convolutional neural network architectures and summarizes the strengths, weaknesses, and innovations of these networks, through which the advancement of neural network architectures shows the great potential of the direction.

show abstract

“…POSTER2:The proposed POSTER2 (Mao et al 2023) aims to improve upon the complex architecture of POSTER, which achieves state-of-the-art performance in facial expression recognition (FER) by combining facial landmark and image features through a two-stream pyramid crossfusion design. POSTER2 reduces computational cost by using a window-based cross-attention mechanism, removing the image-to-landmark branch in the two-stream design, and combining images with landmark's multi-scale features.…”

Section: Visual Featuresmentioning

confidence: 99%

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Zhang¹,

An²,

Zishun³

et al. 2023

Preprint

View full text Add to dashboard Cite

In this paper, we present our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes four sub-challenges of Valence-Arousal (VA) Estimation, Expression (Expr) Classification, Action Unit (AU) Detection and Emotional Reaction Intensity (ERI) Estimation. The 5th ABAW competition focuses on facial affect recognition utilizing different modalities and datasets. In our work, we extract powerful audio and visual features using a large number of sota models. These features are fused by Transformer Encoder and TEMMA. Besides, to avoid the possible impact of large dimensional differences between various features, we design an Affine Module to align different features to the same dimension. Extensive experiments demonstrate that the superiority of the proposed method. For the VA Estimation sub-challenge, our method obtains the mean Concordance Correlation Coefficient (CCC) of 0.6066. For the Expression Classification subchallenge, the average F1 Score is 0.4055. For the AU Detection sub-challenge, the average F1 Score is 0.5296. For the Emotional Reaction Intensity Estimation sub-challenge, the average pearson's correlations coefficient on the validation set is 0.3968. All of the results of four sub-challenges outperform the baseline with a large margin.

show abstract

POSTER++: A simpler and stronger facial expression recognition network

Cited by 11 publications

References 31 publications

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition

A comparative study of deep convolutional neural network-based facial expression regression

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Contact Info

Product

Resources

About