Facial Expressions Recognition in Thermal Images based on Deep Learning Techniques

Elbarawy, Yomna M.; Ghali, Neveen I.; El-Sayed, Rania Salah

doi:10.5815/ijigsp.2019.10.01

Cited by 5 publications

(2 citation statements)

References 13 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most conventional methods rely heavily on feature extraction and classification techniques with significant preprocessing. CNN is a DL technique that can automatically identify and learn significant features from the raw data of images through numerous layers [28].…”

Section: B Fer Using Infrared Datasetmentioning

confidence: 99%

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Naseem,

Lee,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

Facial expression recognition (FER) is one of the best non-intrusive methods for understanding and tracking mood and mental states. In this study, we propose early and late fusion methods to recognize five facial expressions (angry, happy, neutral, sad, and surprised) using different combinations from a publicly available database (VIRI) with visible, infrared, and multispectral dynamic imaging (MSX) images and the (NVIE) database. A distinctive feature is the use of concatenation and combining techniques to combine ResNet-18 with transfer learning (TL) to create a model that is significantly more accurate than individual models. In the early fusion, we concatenated features from the modalities and classified facial expressions (FEs). In the late fusion, we combined the outputs of the modalities using weighted sums. For this purpose, we used different weighting factors depending on the accuracy of the individual models. The experimental results demonstrated that the proposed model outperformed the previous works by providing an accuracy of 83.33% when we trained the model (1-step training). Through further fine-tuning (3-step training), we obtained an improved performance of 84.44%. We conducted additional experiments by combining them with another modality (MSX) available in the database. By performing experiments with an additional modality (MSX), we obtained improved performance, which confirms that the additional modality combined with existing modalities can help improve the performance of fusion models for facial expression recognition. We also experimented by changing the backbones (Vgg-16, ShuffleNetv2, MobileNetv2, and GhostNet) in addition to ResNet-18 for visible and MSX data. ResNet-18 outperformed the other backbones in facial expression recognition for visible and MSX data.

show abstract

Section: B Fer Using Infrared Datasetmentioning

confidence: 99%

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Naseem,

Lee,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Well-known examples of such models are Restricted Boltzmann machines and Deep Belief Networks [9,10] as well as autoencoder neural networks [11,12,13]. In the latter models the encoding and generating components are combined into a single feed-forward network, with a training process for both encoding and generative parts based on minimization of the deviation of regenerated distribution form the original one via one of the loss backpropagation methods, such as stochastic gradient descent methods [14,15] and others.…”

Section: A Literature Reviewmentioning

confidence: 99%

Categorization in Unsupervised Generative Selflearning Systems

Dolgikh¹

2021

IJMECS

View full text Add to dashboard Cite

In this study the authors investigated the connections between the training processes of unsupervised neural network models with self-encoding and regeneration and the information structure in the representations created by such models. We propose theoretical arguments leading to conclusions, confirmed by previously published experimental results that unsupervised representations obtained under certain constraints in training compliant with Bayesian inference principle, favor configurations with better categorization of hidden concepts in the observable data. The results provide an important connection between training of unsupervised machine learning models and the structure of representations created by them and can be used in developing new methods and approaches in self-learning as well as provide insights into common principles underlying the emergence of intelligence in machine and biologic systems.

show abstract

Facial emotion detection using thermal and visual images based on deep learning techniques

Rashmi

Snekhalatha

Salvador

et al. 2023

The Imaging Science Journal

View full text Add to dashboard Cite

Facial Expressions Recognition in Thermal Images based on Deep Learning Techniques

Cited by 5 publications

References 13 publications

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Categorization in Unsupervised Generative Selflearning Systems

Facial emotion detection using thermal and visual images based on deep learning techniques

Contact Info

Product

Resources

About