Automated facial expression recognition using exemplar hybrid deep feature generation technique

Bayğın, Mehmet; Tuncer, Ilknur; Doğan, Şengül; Barua, Prabal Datta; Tuncer, Türker; Cheong, Kang Hao; Acharya, U. Rajendra

doi:10.1007/s00500-023-08230-9

Cited by 8 publications

(1 citation statement)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When compared to other research, the proposed models are competitive, often outperforming or matching other state-of-the-art results. The model of Dada et al 53 , Lasri et al 55 , and Baygin et al 56 stand out with slightly higher metrics on JAFFE. However, it should be noted that the proposed models are trained and tuned using the pre-processing and hyperparameters specifically tailored for Emognition dataset, while the existing models were developed and trained for JAFFE and KDEF datasets.…”

Section: Resultsmentioning

confidence: 82%

Image-based facial emotion recognition using convolutional neural network on emognition dataset

Agung,

Rifai,

Wijayanto

2024

Sci Rep

View full text Add to dashboard Cite

Detecting emotions from facial images is difficult because facial expressions can vary significantly. Previous research on using deep learning models to classify emotions from facial images has been carried out on various datasets that contain a limited range of expressions. This study expands the use of deep learning for facial emotion recognition (FER) based on Emognition dataset that includes ten target emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, sadness, and neutral. A series of data preprocessing was carried out to convert video data into images and augment the data. This study proposes Convolutional Neural Network (CNN) models built through two approaches, which are transfer learning (fine-tuned) with pre-trained models of Inception-V3 and MobileNet-V2 and building from scratch using the Taguchi method to find robust combination of hyperparameters setting. The proposed model demonstrated favorable performance over a series of experimental processes with an accuracy and an average F1-score of 96% and 0.95, respectively, on the test data.

show abstract

Section: Resultsmentioning

confidence: 82%

Image-based facial emotion recognition using convolutional neural network on emognition dataset

Agung,

Rifai,

Wijayanto

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

FacialCueNet: unmasking deception - an interpretable model for criminal interrogation using facial expressions

Nam,

Kim,

Bark

et al. 2023

Appl Intell

View full text Add to dashboard Cite

Polygraphs are used in criminal interrogations to detect deception. However, polygraphs can be difficult to administer under circumstances that prevent the use of biosensors. To address the shortcomings of the biosensors, deception-detection technology without biosensors is needed. We propose a deception-detection method, FacialCueNet, which is a multi-modal network that utilizes both facial images and facial cues based on deep-learning technology. FacialCueNet incorporates facial cues that indicate deception, such as action-unit frequency, symmetry, gaze pattern, and micro-expressions extracted from videos. Additionally, the spatial-temporal attention module, based on convolutional neural network and convolutional long short-term memory, is applied to FacialCueNet to provide interpretable information from interrogations. Because our goal was developing an algorithm applicable to criminal interrogations, we trained and evaluated FacialCueNet using the DDCIT dataset, which was collected using a data acquisition protocol similar to those used in actual investigations. To compare deception-detection performance with state-of-the-art works, a public dataset was also used. As a result, the mean deception-detection F1 score using the DDCIT dataset was 81.22%, with an accuracy of 70.79%, recall of 0.9476, and precision of 0.7107. When evaluating against the public database, our method demonstrated an evaluation accuracy of 88.45% and achieved an AUC of 0.9541, indicating a improvement of 1.25% compared to the previous results. We also present interpretive results of deception detection by analyzing the influence of spatial and temporal factors. These results show that FacialCueNet has the potential to detect deception using only facial videos. By providing interpretation of predictions, our system could be useful tool for criminal interrogation.

show abstract

What Happens in Face During a Facial Expression? Using Data Mining Techniques to Analyze Facial Expression Motion Vectors

Roshanzamir,

Jafari,

Alizadehsani

et al. 2024

Inf Syst Front

View full text Add to dashboard Cite

Automated facial expression recognition using exemplar hybrid deep feature generation technique

Cited by 8 publications

References 53 publications

Image-based facial emotion recognition using convolutional neural network on emognition dataset

Image-based facial emotion recognition using convolutional neural network on emognition dataset

FacialCueNet: unmasking deception - an interpretable model for criminal interrogation using facial expressions

What Happens in Face During a Facial Expression? Using Data Mining Techniques to Analyze Facial Expression Motion Vectors

Contact Info

Product

Resources

About