Micro-expression recognition with attention mechanism and region enhancement

Wang, Yi; Zheng, Shengping; Sun, Xiao; Guo, Dan; Lang, Junjie

doi:10.1007/s00530-022-00934-6

Cited by 3 publications

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, it has major implications when it comes to high-risk situations including criminal investigation, social interactions, national security, and business negotiations ( O’Sullivan et al, 2009 ). It is less accurate to recognize micro-expressions ( Ben et al, 2021 ; Tran et al, 2021 ; Zhou et al, 2021 ; Bisogni et al, 2022 ; Wang et al, 2022 ; Zhu et al, 2022 ; Wei et al, 2022a ) than facial expressions ( Shao and Qian, 2019 ; Li and Deng, 2020 ; Chowdary et al, 2021 ; Bisogni et al, 2022 ). The current research on micro-expression recognition is still correlating datasets collected the laboratory environments, which is not enough for application in high-risk scenes.…”

Section: Introductionmentioning

confidence: 99%

Multi-scale fusion visual attention network for facial micro-expression recognition

et al. 2023

View full text Add to dashboard Cite

IntroductionMicro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest.MethodsThis paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the micro-expression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for micro-expression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model.ResultsThe proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.DiscussionThis paper proposed MFVAN model is the first to combine image generation with visual attention mechanisms to solve the combination challenge problem of individual identity attribute interference and low-intensity facial muscle movements. Meanwhile, the MFVAN model reveal the impact of individual attributes on the localization of local ROIs. The experimental results show that a multi-scale fusion visual attention network contributes to micro-expression recognition.

show abstract