Facial Expression Recognition In-the-Wild with Deep Pre-trained Models

“…The thresholds were estimated based on semi-supervised learning. The SMMEmotionNet was used to extract facial embeddings for an ensemble in the solution of the 6th place [19]. At the same time, the hybrid CNN (Convolutional Neural Network)-Transformer [33] with a fusion of ResNet-18 and a spatial transformer took the 5th place.…”

Section: Related Workmentioning

confidence: 99%

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Savchenko¹

2021

2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY)

124

View full text Add to dashboard Cite

In this paper, we describe the results of the HSEmotion team in two tasks of the seventh Affective Behavior Analysis in-thewild (ABAW) competition, namely, multi-task learning for simultaneous prediction of facial expression, valence, arousal, and detection of action units, and compound expression recognition. We propose an efficient pipeline based on frame-level facial feature extractors pre-trained in multi-task settings to estimate valence-arousal and basic facial expressions given a facial photo. We ensure the privacy-awareness of our techniques by using the lightweight architectures of neural networks, such as MT-EmotiDDAMFN, MT-EmotiEffNet, and MT-EmotiMobileFaceNet, that can run even on a mobile device without the need to send facial video to a remote server. It was demonstrated that a significant step in improving the overall accuracy is the smoothing of neural network output scores using Gaussian or box filters. It was experimentally demonstrated that such a simple post-processing of predictions from simple blending of two top visual models improves the F1-score of facial expression recognition up to 7%. At the same time, the mean Concordance Correlation Coefficient (CCC) of valence and arousal is increased by up to 1.25 times compared to each model's frame-level predictions. As a result, our final performance score on the validation set from the multi-task learning challenge is 4.5 times higher than the baseline (1.494 vs 0.32).

show abstract

Edge Computing with Deep Learning and Internet of Things for Recognising and Predicting Students Emotions and Mental Health

Nowfal,

Ayasrah,

Sadu

et al. 2024

JMC

View full text Add to dashboard Cite

A person's Mental Health (MH) dramatically influences their complete evolution in life, including their cognitive, emotional, and psychomotor components. A person with good MH is content with life and can be creative, learn new things, and take risks to accomplish more significant objectives. Currently, college students are dealing with MH concerns for various causes, which affect their academic performance and significantly contribute to poor academic results. Therefore, encouraging MH in college students presents a significant problem for educators, parents, teacher educators, and governments. Adolescence is a crucial and delicate time characterized by considerable physical, emotional, social, and religious changes. The physical, social, and psychological facets of an individual's growth are laid out in this period, with mental health as a crucial factor in promoting these gains. Therefore, it becomes crucial for researchers to use Deep Learning (DL) algorithms to study the association between MH and vital psychological characteristics, such as emotional intelligence, personality traits, and intelligence. The personal aspects, namely personality, emotional intelligence, and MH are all related ideas that influence one another. Individuals must have mental well-being and emotional harmony to have a good personality. The current study uses DL techniques to investigate the relationship between college students' MH, emotional intelligence, and personality features. To perform a thorough study on emotion identification and Mental Health Prediction (MHP) among college students, this project investigates the integration of edge computing enabled by the Internet of Things (IoT) in the context of intelligent systems. Innovative treatments are urgently needed due to this population's rising prevalence of MH issues. This paper aims to continuously monitor and predict college students' MH using Edge Computing (EC) and IoT technology.

show abstract

Facial Expression Recognition In-the-Wild with Deep Pre-trained Models

Cited by 3 publications

References 29 publications

Facial Expression Recognition Using Pre-trained Architectures

Facial Expression Recognition Using Pre-trained Architectures

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

Edge Computing with Deep Learning and Internet of Things for Recognising and Predicting Students Emotions and Mental Health

Contact Info

Product

Resources

About