Facial Expression Recognition Based on Squeeze Vision Transformer

Kim, Sangwon; Nam, Jae-Yeal; Ko, Byoung Chul

doi:10.3390/s22103729

Cited by 28 publications

(10 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, with this method, channel attention is used only as a means for global feature extraction, and the experiment is limited to the FER 2013 dataset. Kim et al [35] proposed a squeeze vision transformer, a method to reduce computational complexity by reducing the number of feature dimensions while increasing FER performance. In this method, visual tokens and landmark heatmap-based local tokens are combined such that global and local patch characteristics can be maintained at the same time.…”

Section: Fer In the Wild With Attentionmentioning

confidence: 99%

“…A comparison method was selected among SOTA FER studies specialised for wild FER with 1) a patch-based attention CNN mechanism (pACNN) [23], 2) adversarial graph representation (AGR) [55], 3) region attention network (RAN) [32], 4) feature decomposition and reconstruction learning (FDRL) [28], 5) EfficientFace [56], 6) deep attentive centre loss (DACL) [20], 7) latent distribution mining and pairwise uncertainty estimation (DMUE) [57], 8) relative uncertainty learning (RUL) [58], 9) FER with visual transformers with feature fusion (VTFF) [59], 10) squeeze vision transformer (Squeeze-ViT) [35], 11) FER through the meta (Face2Exp) [29], and 12) vision transformer with attention pooling (APViT) [60]. As shown in Table 1, the proposed method combining the face graph and a GCN showed the best performance in all three datasets, except AffectNet-8.…”

Section: A Performance Comparison With State-of-art Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Facial Expression Recognition in the Wild Using Face Graph and Attention

Kim

Lee

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Facial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expressions, and we propose an algorithm for recognizing facial expressions using a graph convolutional network (GCN). We first generated an attention map that can highlight action units to extract important facial expression features from faces in the wild. After feature extraction, a face graph is constructed by combining the attention map with face patches, and changes in expression in the wild are recognized using a GCN. Through comparative experiments conducted using both lab-controlled and wild datasets, we prove that the proposed method is the most suitable FER approach for use with image datasets captured in the wild and those under well-controlled indoor conditions.

show abstract

Section: Fer In the Wild With Attentionmentioning

confidence: 99%

Section: A Performance Comparison With State-of-art Approachesmentioning

confidence: 99%

Facial Expression Recognition in the Wild Using Face Graph and Attention

Kim

Lee

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Influenced by the vision transformer, Xue et al [51] designed the first transformer-based FER network to model long-range dependencies for FER. Kim et al [24] improved the vision transformer (ViT) to combine both global and local features so that ViT can be adapted to FER task.…”

Section: Introductionmentioning

confidence: 99%

POSTER++: A simpler and stronger facial expression recognition network

Mao¹,

Xu²,

Yin³

et al. 2023

Preprint

View full text Add to dashboard Cite

Facial expression recognition (FER) plays an important role in a variety of real-world applications such as human-computer interaction. POSTER V1 achieves the state-of-the-art (SOTA) performance in FER by effectively combining facial landmark and image features through two-stream pyramid cross-fusion design. However, the architecture of POSTER V1 is undoubtedly complex. It causes expensive computational costs. In order to relieve the computational pressure of POSTER V1, in this paper, we propose POSTER V2. It improves POSTER V1 in three directions: cross-fusion, two-stream, and multiscale feature extraction. In cross-fusion, we use windowbased cross-attention mechanism replacing vanilla crossattention mechanism. We remove the image-to-landmark branch in the two-stream design. For multi-scale feature extraction, POSTER V2 combines images with landmark's multi-scale features to replace POSTER V1's pyramid design. Extensive experiments on several standard datasets show that our POSTER V2 achieves the SOTA FER performance with the minimum computational cost. For example, POSTER V2 reached 92.21% on RAF-DB, 67.49% on AffectNet (7 cls) and 63.77% on AffectNet (8 cls), respectively, using only 8.4G floating point operations (FLOPs) and 43.7M parameters (Param). This demonstrates the effectiveness of our improvements. The code and models are available at https://github.com/Talented-Q/ POSTER_V2.

show abstract

“…Sangwon Kim et al proposed the Squeeze Vit method to study facial expression recognition with squeezed vision converters. They set the training set (validation set + training set) and the test set of the FER dataset at 80.0% and 20.0%, respectively [8]. Xiangkui Jiang et al proposed a smoking behavior detection method based on YOLOv5 [9], using a homemade smoking behavior dataset for training and setting the proportion of training set to test set to 7:3.…”

Section: Introductionmentioning

confidence: 99%

A TEDE Algorithm Studies the Effect of Dataset Grouping on Supervised Learning Accuracy

Wang

Song

et al. 2023

Electronics

View full text Add to dashboard Cite

Datasets are the basis for research on deep learning methods in computer vision. The impact of the percentage of training sets in a dataset on the performance of neural network models needs to be further explored. In this paper, a twice equal difference enumeration (TEDE) algorithm is proposed to investigate the effect of different training set percentages in the dataset on the performance of the network model, and the optimal training set percentage is determined. By selecting the Pascal VOC dataset and dividing it into six different datasets from largest to smallest, and then dividing each dataset into the datasets to be analyzed according to five different training set percentages, the YOLOv5 convolutional neural network is used to train and test the 30 datasets to determine the optimal neural network model corresponding to the training set percentages. Finally, tests were conducted using the Udacity Self-Driving dataset with a self-made Tire Tread Defects (TTD) dataset. The results show that the network model performance is superior when the training set accounts for between 85% and 90% of the overall dataset. The results of dataset partitioning obtained by the TEDE algorithm can provide a reference for deep learning research.

show abstract

Facial Expression Recognition Based on Squeeze Vision Transformer

Cited by 28 publications

References 35 publications

Facial Expression Recognition in the Wild Using Face Graph and Attention

Facial Expression Recognition in the Wild Using Face Graph and Attention

POSTER++: A simpler and stronger facial expression recognition network

A TEDE Algorithm Studies the Effect of Dataset Grouping on Supervised Learning Accuracy

Contact Info

Product

Resources

About