Inceptr: micro-expression recognition integrating inception-CBAM and vision transformer

Zhou, Haoliang; Huang, Shucheng; Xu, Yuqiao

doi:10.1007/s00530-023-01164-0

Cited by 13 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Through fusing two different semantic features, they obtain hybrid features with local edge information and global appearance information. Zhou et al [20] addressed Transformer to extract subtle facial motion features and to model the global relationship among Patches. They took the rich relationship among different rich relationships within patches to mine discriminative features.…”

Section: Transformer In Fermentioning

confidence: 99%

Facial Expression Recognition with Enhanced Relation-Aware Attention and Cross-Feature Fusion transformer

DONG,

Wang,

et al. 2024

Preprint

View full text Add to dashboard Cite

Face expression recognition(FER) is an important research branch in the field of the computer vision neighborhood. Three prevalent problems in FER tasks that severely impact recognition rates are inter-class similarity, intra-class differences, and facial occlusion issues. Although there have been studies that address some of these issues, none of them can adequately address all three issues in a unified framework. In this paper, we propose a novel dual-branch structure of enhanced relation-aware attention and cross-feature fusion transformer network to comprehensively solve all three issues. Specifically, we design the Enhanced Relation-Aware Attention module to maximize the exploration of more local expression features. At the same time, the Transformer Perceptual Encoder module is adopted to establishing the contextual relationship between individual patches under global information. This greatly alleviates the inter-class similarity problem and the facial occlusion and facial pose transformation problems. On the basis of a dual branch structure, we extract facial image features using facial landmarks features to guide them and design Cross-Feature Fusion Transformer module to deeply cross-fuse two different semantic features. Experiments are performed and results show that our method can greatly alleviated intra-class difference problem with comparison of several traditional methods on three commonly used datasets.

show abstract

Section: Transformer In Fermentioning

confidence: 99%

Facial Expression Recognition with Enhanced Relation-Aware Attention and Cross-Feature Fusion transformer

DONG,

Wang,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Overcoming the limitations of existing vision transformers, this novel approach balances local spatial relationships and global dependencies. Achieving exemplary performance on datasets such as CASME-I, CASME-II, and SAMM, this methodology integrates convolutional layers and transformers for improved spatial information and feature extraction [16] [17].…”

Section: Architectural Innovationsmentioning

confidence: 99%

Facial Expression Recognition Based on Deep Learning: A Review

Jajan,

Abdulazeez

2024

ijcs

View full text Add to dashboard Cite

This review paper provides a comprehensive analysis of recent advancements in Facial Expression Recognition (FER) through various deep learning models. Seven state-of-the-art models are scrutinized, each offering unique contributions to the field. The MBCC-CNN model demonstrates improved recognition rates on diverse datasets, addressing the challenges of facial expression recognition through multiple branches and cross-connected convolutional neural networks. The Deep Graph Fusion model introduces a novel approach for predicting viewer expressions from videos, showcasing superior performance on the EEV database. Multimodal emotion recognition is explored in the EEG and facial expression fusion model, achieving high accuracy on the DEAP dataset. The Spark-based LDSP-TOP descriptor, coupled with a 1-D CNN and LSTM Autoencoder, excels in capturing temporal dynamics for facial expression understanding. Vision transformers for micro-expression recognition exhibit outstanding accuracy on datasets like CASMEI, CASME-II, and SAMM. Additionally, a hierarchical deep learning model is proposed for evaluating teaching states based on facial expressions. Lastly, a visionary transformer model achieves remarkable recognition accuracy of 100% on SAMM dataset, showcasing the potential of combining convolutional and transformer architectures. This review synthesizes key findings, highlights model performances, and outlines directions for future research in FER.

show abstract

“…Zhu et al fuse CNN with Transformer, adopting a sparse self-attention mechanism to extract a sparse representation of feature maps [22], which can reduce the network complexity and improve the recognition performance. The model in [23] integrates inception-CBAM with a Vision Transformer for micro-expression recognition. A recent study uses a combination of CNN and Transformer to fuse local and global facial information in facial MEs [24].…”

Section: Related Workmentioning

confidence: 99%

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Xie,

Zhao

2024

Electronics

View full text Add to dashboard Cite

A micro-expression (ME), as a spontaneous facial expression, usually occurs instantaneously and is difficult to disguise after an emotion-evoking event. Numerous convolutional neural network (CNN)-based models have been widely explored to recognize MEs for their strong local feature representation ability on images. However, the main drawback of the current methods is their inability to fully extracting holistic contextual information from ME images. To achieve efficient ME learning representation from diverse perspectives, this paper uses Transformer variants as the main backbone and the dual-branch architecture as the main framework to extract meaningful multi-modal contextual features for ME recognition (MER). The first branch leverages an optical flow operator to facilitate the motion information extraction between ME sequences, and the corresponding optical flow maps are fed into the Swin Transformer to acquire motion–spatial representation. The second branch directly sends the apex frame in one ME clip to Mobile ViT (Vision Transformer), which can capture the local–global features of MEs. More importantly, to achieve the optimal feature stream fusion, a CAB (cross attention block) is designed to interact the feature extracted by each branch for adaptive learning fusion. The extensive experimental comparisons on three publicly available ME benchmarks show that the proposed method outperforms the existing MER methods and achieves an accuracy of 81.6% on the combined database.

show abstract

Inceptr: micro-expression recognition integrating inception-CBAM and vision transformer

Cited by 13 publications

References 43 publications

Facial Expression Recognition with Enhanced Relation-Aware Attention and Cross-Feature Fusion transformer

Facial Expression Recognition with Enhanced Relation-Aware Attention and Cross-Feature Fusion transformer

Facial Expression Recognition Based on Deep Learning: A Review

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Contact Info

Product

Resources

About