Recognition and Intensity Estimation of Facial Expression Using Ensemble Classifiers

Nomiya, Hiroki; Sakaue, Shota; Hochin, Teruhisa

doi:10.2991/ijndc.2016.4.4.1

Cited by 5 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As this miniature sensor has small dimensions of 2.7 x 3.2 x 1.4 mm (W x L x H), 40 units are densely arranged at sampling points on the face capture mask, as shown in Figure 5. Following the standard practice of face recognition [24,25,33], we picked eyebrows, cheeks and around eyes and mouth as the sampling points of photo reflective sensors. We ensure that the sensors do not come into contact with the eyes, nose and mouth of the mask wearer.…”

Section: Facial Expression Identification Methodsmentioning

confidence: 99%

Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays

Takegawa

Tokuda²,

Umezawa

et al. 2020

2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

This paper presents a thin digital full-face mask display that can reflect an entire facial expression of a user onto an avatar to support augmented face-to-face communication in real environments. Although camera-based facial expression recognition technology has enabled people to augment their faces with avatars, application was limited to face-to-face communication in virtual environments. To enable digital facial augmentation with an avatar in a real space, we propose a digital face mask display system that integrates a lightweight flexible display with a thin facial expression recognition system. The thin wearable facial expression recognition system was implemented with photo reflective sensor arrays which can measure facial expressions at 40 feature points distributed across an entire face. We investigated a ten-class facial expression identification model based on an SVM training algorithm. The trained model achieved an average accuracy of 79% when identifying the facial expressions of multiple users. User experiments indicated that the proposed thin digital full-face mask display allows the wearer to control the facial expression of the avatar with a fast response rate and create a positive sense of self-agency and self-ownership toward the augmented avatar face.

show abstract

Section: Facial Expression Identification Methodsmentioning

confidence: 99%

Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays

Takegawa

Tokuda²,

Umezawa

et al. 2020

2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

show abstract

“…To improve the representation ability of a given approach, handcraft features are often designed with high feature dimensions and large amounts of redundant information, which may lead to dimensional disaster [12]. In response, machine learning approaches such as ensemble learning [13], kernel learning [5], and dictionary learning [14,15] have been developed to adaptively select suitable features for dimension reduction and simultaneously learn the expression intensity model with the selected features.…”

Section: Facial Expression Intensity Estimationmentioning

confidence: 99%

Facial expression intensity estimation using label-distribution-learning-enhanced ordinal regression

Xu,

Wang,

Chen

et al. 2024

Multimedia Systems

View full text Add to dashboard Cite

Facial expression intensity estimation has promising applications in health care and affective computing, such as monitoring patients' pain feelings. However, labeling facial expression intensity is a specialized and time-consuming task. To overcome the lack of labeled data, a variety of ordinal regression (OR) models have been presented to estimate the relative intensity of each expression image within a sequence in an unsupervised setting. However, these models cannot estimate absolute intensity without actual intensity labels. To overcome this problem, this paper introduces the label-distribution-learning-enhanced OR (LDL-EOR) approach for facial expression intensity estimation. LDL-EOR learns relative intensity with OR and absolute intensity by using the label distribution to resist the noisy labels caused by the biases of manual and automatic labeling. This design aims to improve the accuracy of absolute intensity estimation while keeping the cost of manual labeling low. The label distribution is converted into a continuous intensity value by calculating the mathematical expectation, which makes the prediction results meet both relative and absolute intensity constraints. Ensuring the feasibility of LDL-EOR in different supervised settings, this paper presents a unified label distribution generation framework to automatically relabel training data frame by frame. The generated soft labels are used to supervise the LDL-EOR model and enhance its robustness to the noise existing in the original labels. Numerous experiments were conducted on three public expression datasets (CK+, BU-4DFE, and PAIN) to validate the superiority of LDL-EOR relative to other state-of-the-art approaches.

show abstract

“…The few works on emotion recognition and intensity estimation are categorised in Kamarol et al (2017) as: the distance-based ( Verma et al, 2005 ), the cluster-based ( Quan, Qian & Ren, 2014 ), the graphical-based ( Valstar & Pantic, 2012 ) and the regression-based ( Nomiya, Sakaue & Hochin, 2016 ) methods. As stated earlier, our focus is on recent deep learning approaches to emotion recognition and intensity estimation.…”

Section: Related Workmentioning

confidence: 99%

Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation

Ekundayo¹,

Viriri²

2021

PeerJ Computer Science

View full text Add to dashboard Cite

Facial Expression Recognition (FER) has gained considerable attention in affective computing due to its vast area of applications. Diverse approaches and methods have been considered for a robust FER in the field, but only a few works considered the intensity of emotion embedded in the expression. Even the available studies on expression intensity estimation successfully assigned a nominal/regression value or classified emotion in a range of intervals. Most of the available works on facial expression intensity estimation successfully present only the emotion intensity estimation. At the same time, others proposed methods that predict emotion and its intensity in different channels. These multiclass approaches and extensions do not conform to man heuristic manner of recognising emotion and its intensity estimation. This work presents a Multilabel Convolution Neural Network (ML-CNN)-based model, which could simultaneously recognise emotion and provide ordinal metrics as the intensity estimation of the emotion. The proposed ML-CNN is enhanced with the aggregation of Binary Cross-Entropy (BCE) loss and Island Loss (IL) functions to minimise intraclass and interclass variations. Also, ML-CNN model is pre-trained with Visual Geometric Group (VGG-16) to control overfitting. In the experiments conducted on Binghampton University 3D Facial Expression (BU-3DFE) and Cohn Kanade extension (CK+) datasets, we evaluate ML-CNN’s performance based on accuracy and loss. We also carried out a comparative study of our model with some popularly used multilabel algorithms using standard multilabel metrics. ML-CNN model simultaneously predicts emotion and intensity estimation using ordinal metrics. The model also shows appreciable and superior performance over four standard multilabel algorithms: Chain Classifier (CC), distinct Random K label set (RAKEL), Multilabel K Nearest Neighbour (MLKNN) and Multilabel ARAM (MLARAM).

show abstract

Recognition and Intensity Estimation of Facial Expression Using Ensemble Classifiers

Cited by 5 publications

References 7 publications

Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays

Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays

Facial expression intensity estimation using label-distribution-learning-enhanced ordinal regression

Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation

Contact Info

Product

Resources

About