Personalized models for facial emotion recognition through transfer learning

Abstract

Emotions represent a key aspect of human life and behavior. In recent years, automatic recognition of emotions has become an important component in the fields of affective computing and human-machine interaction. Among many physiological and kinematic signals that could be used to recognize emotions, acquiring facial expression images is one of the most natural and inexpensive approaches. The creation of a generalized, inter-subject, model for emotion recognition from facial expression is still a challenge, due to anatomical, cultural and environmental differences. On the other hand, using traditional machine learning approaches to create a subject-customized, personal, model would require a large dataset of labelled samples. For these reasons, in this work, we propose the use of transfer learning to produce subject-specific models for extracting the emotional content of facial images in the valence/arousal dimensions. Transfer learning allows us to reuse the knowledge assimilated from a large multi-subject dataset by a deep-convolutional neural network and employ the feature extraction capability in the single subject scenario. In this way, it is possible to reduce the amount of labelled data necessary to train a personalized model, with respect to relying just on subjective data. Our results suggest that generalized transferred knowledge, in conjunction with a small amount of personal data, is sufficient to obtain high recognition performances and improvement with respect to both a generalized model and personal models. For both valence and arousal dimensions, quite good performances were obtained (RMSE = 0.09 and RMSE = 0.1 for valence and arousal, respectively). Overall results suggested that both the transferred knowledge and the personal data helped in achieving this improvement, even though they alternated in providing the main contribution. Moreover, in this task, we observed that the benefits of transferring knowledge are so remarkable that no specific active or passive sampling techniques are needed for selecting images to be labelled.

New Breakthroughs and Innovation Modes in English Education in Post-pandemic Era

“…In neural networks, a single deep CNN-based model was proposed to recognize facial expressions (Jain et al, 2019). In automatic recognition of emotions, the authors used migratory learning to generate models of specific subjects to extract emotional content from facial images in the valence/wake dimension (Rescigno et al, 2020). In dual-channel emoticon recognition, machine learning theory and a philosophicalthought-based feature fusion dual-channel emoticon recognition algorithm were proposed (Song, 2021).…”

An Application of a Runtime Epistemic Probabilistic Event Calculus to Decision-making in e-Health Systems

“…Emotion Recognition Module: This module estimates the emotional response of the child, and outputs a valence (Rescigno et al . 2020).…”

Laplacian Nonlinear Logistic Stepwise and Gravitational Deep Neural Classification for Facial Expression Recognition

“…Despite minimizing trainable parameters, the error involved was not concentrated. To address on this aspect, transfer learning was introduced in [5] that with the aid of deep convolutional neural network minimized the amount of labeled data that reducing root mean square error.…”