Adaptive Deep Disturbance-Disentangled Learning for Facial Expression Recognition

Ruan, Delian; Mo, Rongyun; Yan, Yan; Chen, Si; Xue, Jing-Hao; Wang, Hanzi

doi:10.1007/s11263-021-01556-7

Cited by 26 publications

(5 citation statements)

References 81 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by Belghazi et al (2018) and Ruan et al (2022) , we use mutual information to measure the correlation between and . Specifically, in this paper we use Mutual Information Neural Estimator (MINE), which is a neural network based on KL divergence and Donsker–Varadhan representation.…”

Section: Methodsmentioning

confidence: 99%

Decoupling facial motion features and identity features for micro-expression recognition

Xie¹,

Sun²,

Sun³

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Background Micro-expression is a kind of expression produced by people spontaneously and unconsciously when receiving stimulus. It has the characteristics of low intensity and short duration. Moreover, it cannot be controlled and disguised. Thus, micro-expression can objectively reflect people’s real emotional states. Therefore, automatic recognition of micro-expressions can help machines better understand the users’ emotion, which can promote human-computer interaction. What’s more, micro-expression recognition has a wide range of applications in fields like security systems and psychological treatment. Nowadays, thanks to the development of artificial intelligence, most micro-expression recognition algorithms are based on deep learning. The features extracted by deep learning model from the micro-expression video sequences mainly contain facial motion feature information and identity feature information. However, in micro-expression recognition tasks, the motions of facial muscles are subtle. As a result, the recognition can be easily interfered by identity feature information. Methods To solve the above problem, a micro-expression recognition algorithm which decouples facial motion features and identity features is proposed in this paper. A Micro-Expression Motion Information Features Extraction Network (MENet) and an Identity Information Features Extraction Network (IDNet) are designed. By adding a Diverse Attention Operation (DAO) module and constructing divergence loss function in MENet, facial motion features can be effectively extracted. Global attention operations are used in IDNet to extract identity features. A Mutual Information Neural Estimator (MINE) is utilized to decouple facial motion features and identity features, which can help the model obtain more discriminative micro-expression features. Results Experiments on the SDU, MMEW, SAMM and CASME II datasets were conducted, which achieved competitive results and proved the superiority of the proposed algorithm.

show abstract

Section: Methodsmentioning

confidence: 99%

Decoupling facial motion features and identity features for micro-expression recognition

Xie¹,

Sun²,

Sun³

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…The ADDL model in [27] states that the DDL model has two limitations. It can not adaptively choose the disturbance factor while training and the disentanglement process of the disturbance factor are not performed explicitly.…”

Section: B DL Models For Fer With Attentionmentioning

confidence: 99%

Best Fit Activation Functions for Attention Mechanism: Comparison and Enhancement

Alhazmi,

Altahhan

2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Activation functions are one of the critical elements of neural networks that allow them to produce non-linear, fine and complex decision boundaries. Yet, their effects are not very well understood in the context of attention mechanisms. In this paper, we investigate the role of two widely used family of activation functions in conjunction with three attention mechanisms on two widely used image classification models; ResNet50 and MobileNetV2. We modified the structures of these classification models by infusing them with three attention mechanisms, CBAM, BAM, and Triplet Attention. In addition, we equipped them with different activation functions, including ReLU, ELU, and a newly proposed activation function that we call ELU+. The resultant models' performances were examined in the domain of facial expression recognition using three datasets; two lab-controlled, CK+ and JAFFE, and one real-world, FER2013. Compared with the baseline models, our results show a significant increase of up to +30% of models' performance when using the newly proposed AF.

show abstract

“…Hence, with the rapid development of large-scale datasets as well as the advancement in the computational ability, recent studies mainly focus on deep learning as a better alternative. Ruan et al [26], [27] proposed feature decomposition and reconstruction learning for effective facial expression recognition, they divided the facial expressions into groups with different attributes to learn more robust features. Li et al [28] use the attention mechanism to solve the occlusion problem in FER.…”

Section: Related Workmentioning

confidence: 99%

“…WS-LGRN [15] also leverages the attention mechanism to deal with part location and feature fusion problems to better recognize facial expressions. ADDL [27], [35] explores multitask learning and adversarial transfer learning to explicitly disentangling multiple disturbing factors for facial expression recognition. Wang et al [36] propose an adversarial feature learning method to simultaneously focus on pose variations and identity bias in FER.…”

Section: B Feature-level Solutionsmentioning

confidence: 99%

CFNet: A Coarse-to-Fine Learning Strategy for Facial Expression Recognition

Zhang¹

2022

Preprint

View full text Add to dashboard Cite

<p>Facial expression recognition (FER) is a challenging job in Computer Vision due to data uncertainties rooted in the ambiguity of facial expressions. As a complement to current FER studies huddling in data-level or feature-level for suppressing such uncertainties, we propose a simple yet efficient coarse-to-fine learning strategy at task-level inspired by human beings' emotion cognitive mode. Specifically, a child learns quickly whether his behavior is allowed by reading adults' facial expressions for coarse attitude like positive or negative, and then adjusts accordingly via further interpreting its fine sentiment meaning like happy or angry. Similarly, we follow the same divide-and-conquer paradigm and decompose FER into two correlated easier sub-tasks: i) coarse classification for attitude analysis and ii) fine recognition for sentiment interpretation, and build a multi-branch deep network (CFNet) to tackle them in a joint manner. The key idea is to aggregate the discrete universal facial expressions into several coarse groups reflecting attitude tendency based on their empirical projections in the continuous Valence-Arousal (VA) emotion space. However, the coarse classification sub-task is inherently tougher due to its significantly larger intra-class variations compared to the fine recognition sub-task. Such discord could lead to immature learning and degrade the overall performance. To overcome this issue, CFNet leverages a synchronization mechanism to control the learning process via knowledge sharing between both sub-tasks. In addition, a novel center loss is introduced to enhance the discriminative power of the network in extracting compact intra-class representations while preserving intrinsic inter-class relationships. Experiments on three benchmark datasets show that our method achieves state-of-the-art performance, which demonstrates its superiority. The code is available at https://github.com/codpub/CFNet.</p>

show abstract

Adaptive Deep Disturbance-Disentangled Learning for Facial Expression Recognition

Cited by 26 publications

References 81 publications

Decoupling facial motion features and identity features for micro-expression recognition

Decoupling facial motion features and identity features for micro-expression recognition

Best Fit Activation Functions for Attention Mechanism: Comparison and Enhancement

CFNet: A Coarse-to-Fine Learning Strategy for Facial Expression Recognition

Contact Info

Product

Resources

About