Deep Learning Models for Facial Expression Recognition

Sajjanhar, Atul; Wu, Zhaoqi; Wen, Quan

doi:10.1109/dicta.2018.8615843

Cited by 39 publications

(21 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many papers have studied facial expression recognition. There are several techniques on facial expression recognition, but recently deep learning methods have contributed to improving facial expression recognition, with works such as [ 17 , 18 , 19 , 20 , 21 ]. In [ 17 ] a model based on a single deep convolutional neural network (DNN) was proposed, which contained convolution layers and deep residual blocks.…”

Section: Literature Reviewmentioning

confidence: 99%

“…In [ 19 ] a hybrid convolution-recurrent neural network method was used. In [ 20 ] the performance of inception and VGG architectures, which are pre-trained for object recognition, were evaluated and these were compared with VGG-Face, which is pre-trained for face recognition. In [ 21 ] an ensemble of convolutional neural networks with probability-based fusion for facial expression recognition was presented, where the architecture of each CNN was adapted by using the convolutional rectified linear layer as the first layer and multiple hidden layers.…”

Section: Literature Reviewmentioning

confidence: 99%

“…In [ 21 ] an ensemble of convolutional neural networks with probability-based fusion for facial expression recognition was presented, where the architecture of each CNN was adapted by using the convolutional rectified linear layer as the first layer and multiple hidden layers. Most of papers work with one or several datasets separately in order to improve current results [ 17 , 18 , 19 , 20 , 21 ]. That is training and testing sets belonging to the same dataset, but when we test with other databases different from the training set (a cross-dataset approach), the results can be very low [ 22 , 23 , 38 ].…”

Section: Literature Reviewmentioning

confidence: 99%

“…Recently, a similar paper [ 48 ] proposed a novel deep convolutional neural network (CNN) architecture previously trained for the recognition of emotions in unrestricted environments. The difficulty of this work is that a CNN is able to recognize facial expressions on the wild, since the majority of works are trained and tested with the same databases [ 17 , 18 , 19 , 20 , 21 ]. Unlike [ 48 ], we test our own trained CNN using the same social robot (NAO) with more participants and we validate the facial expression images with 10 experts instead of three as suggested in [ 48 ].…”

Section: Literature Reviewmentioning

confidence: 99%

“…This is when the training set and test set come from different databases. If the training set and test set come from the same database, the classification accuracy is more satisfactory [ 17 , 18 , 19 , 20 , 21 ] than if they come from different databases [ 22 , 23 ], where the classification results may decrease up to a 49%.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Using a Social Robot to Evaluate Facial Expressions in the Wild

Perales

Buades

2020

Sensors

View full text Add to dashboard Cite

In this work an affective computing approach is used to study the human-robot interaction using a social robot to validate facial expressions in the wild. Our global goal is to evaluate that a social robot can be used to interact in a convincing manner with human users to recognize their potential emotions through facial expressions, contextual cues and bio-signals. In particular, this work is focused on analyzing facial expression. A social robot is used to validate a pre-trained convolutional neural network (CNN) which recognizes facial expressions. Facial expression recognition plays an important role in recognizing and understanding human emotion by robots. Robots equipped with expression recognition capabilities can also be a useful tool to get feedback from the users. The designed experiment allows evaluating a trained neural network in facial expressions using a social robot in a real environment. In this paper a comparison between the CNN accuracy and human experts is performed, in addition to analyze the interaction, attention and difficulty to perform a particular expression by 29 non-expert users. In the experiment, the robot leads the users to perform different facial expressions in motivating and entertaining way. At the end of the experiment, the users are quizzed about their experience with the robot. Finally, a set of experts and the CNN classify the expressions. The obtained results allow affirming that the use of social robot is an adequate interaction paradigm for the evaluation on facial expression.

show abstract