Improvement of emotion recognition from facial images using deep learning and early stopping cross validation

Bentoumi, Mohamed; Daoud, M.; Benaouali, Mohamed; Ahmed, Abdelmalik Taleb

doi:10.1007/s11042-022-12058-0

Cited by 29 publications

(32 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies in Facial Expressions Recognition (FER) and Pain Assessment have introduced multiple cutting-edge models that achieve an acceptable performance. These models have tackled various challenges related to performance improvement by leveraging state-of-the-art Convolutional Neural Networks (CNN) [17,18] or by adopting Long Short-Term Memory (LSTM) [19]. Moreover, researchers have explored the impact of various techniques such as augmentation [20] and batch normalization [21].…”

Section: Related Workmentioning

confidence: 99%

“…While most studies focus on improving recognition accuracy, challenges such as poses, occlusion, and illumination demand further investigation and more efficient solutions. In order to compare the SAFEPA with the state-of-the-art, we have selected four recent studies [17,[24][25][26] that use CNN, HOG, and LBP for FER on the same datasets as our study. Additionally, we included three more recent studies that focus on facial expression-based pain assessment [3,22,23].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

Alghamdi

Alaghband

2023

Applied Sciences

View full text Add to dashboard Cite

Accurately assessing the intensity of pain from facial expressions captured in videos is crucial for effective pain management and critical for a wide range of healthcare applications. However, in uncontrolled environments, detecting facial expressions from full left and right profiles remains a significant challenge, and even the most advanced models for recognizing pain levels based on facial expressions can suffer from declining performance. In this study, we present a novel model designed to overcome the challenges posed by full left and right profiles—Sparse Autoencoders for Facial Expressions-based Pain Assessment (SAFEPA). Our model utilizes Sparse Autoencoders (SAE) to reconstruct the upper part of the face from the input image, and feeds both the original image and the reconstructed upper face into two pre-trained concurrent and coupled Convolutional Neural Networks (CNNs). This approach gives more weight to the upper part of the face, resulting in superior recognition performance. Moreover, SAFEPA’s design leverages CNNs’ strengths while also accommodating variations in head poses, thus eliminating the need for face detection and upper-face extraction preprocessing steps needed in other models. SAFEPA achieves high accuracy in recognizing four levels of pain on the widely used UNBC-McMaster shoulder pain expression archive dataset. SAFEPA is extended for facial expression recognition, where we show it to outperform state-of-the-art models in recognizing seven facial expressions viewed from five different angles, including the challenging full left and right profiles, on the Karolinska Directed Emotional Faces (KDEF) dataset. Furthermore, the SAFEPA system is capable of processing BioVid Heat Pain datasets with an average processing time of 17.82 s per video (5 s in length), while maintaining a competitive accuracy compared to other state-of-the-art pain detection systems. This experiment demonstrates its applicability in real-life scenarios for monitoring systems. With SAFEPA, we have opened new possibilities for accurate pain assessment, even in challenging situations with varying head poses.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

Alghamdi

Alaghband

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…To test the generalizability of the proposed LEmo method, we choose two different datasets; i) CK+ and ii) KDEF. These datasets are chosen as they have recently been used in a wide number of studies [57,58,59,60,61].…”

Section: Datasetmentioning

confidence: 99%

An anti-attack method for emotion categorization from images

Shehu

Browne

Eisenbarth

2022

Applied Soft Computing

View full text Add to dashboard Cite

“…The extracted features are used as inputs for the three classifiers, namely, SVM, KNN and decision tree (DT). In this study, to objectively highlight the performance of our procedure, we assess classification performance using three evaluation metrics, namely accuracy, sensitivity and specificity [9][10].…”

Section: • Classificationmentioning

confidence: 99%

“…These metrics are complementary and summarize the performance of a classifier by taking into account all the components of the confusion matrix. The latter is composed of 4 values (Table I) in the case of a two-class classification issue (positive class vs. negative class) [9][10].…”

mentioning

confidence: 99%

Segmentation and classification of benign and malignant breast tumors via texture characterization from ultrasound images

Benaouali

Bentoumi

Touati

et al. 2022

2022 7th International Conference on Image and Signal Processing and Their Applications (ISPA)

Self Cite

View full text Add to dashboard Cite

The present paper deals with breast tumors classification from ultrasound images. The proposed procedure consists of four steps, namely preprocessing, segmentation, feature extraction and classification. To improve the quality of ultrasound images, the preprocessing step consists of anisotropic filtering and histogram equalization that are performed on the original images. The segmentation is performed on the preprocessed images using the Level Set method that allows to extract the region of interest (ROI) and to reduce its size at the same time. Two feature extraction methods are used in this work namely, the local binary pattern (LBP) method and the histogram of oriented gradients (HOG) method. The two methods (LBP and HOG) are techniques of textures analysis and allow to characterize the ROI. The extracted feature sets constitute the inputs for three classifiers namely, support vector machines (SVM), k-nearest neighbors (KNN) and decision trees (DT). In this work, the best results are obtained by the concatenation of the two feature vectors namely LBP and HOG associated to the SVM classifier. This allows to achieve an accuracy of 96%, a sensitivity of 97% and a specificity of 94%.

show abstract

Improvement of emotion recognition from facial images using deep learning and early stopping cross validation

Cited by 29 publications

References 44 publications

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

An anti-attack method for emotion categorization from images

Segmentation and classification of benign and malignant breast tumors via texture characterization from ultrasound images

Contact Info

Product

Resources

About