Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013

Giannopoulos, Panagiotis; Perikos, Isidoros; Hatzilygeroudis, Ioannis

doi:10.1007/978-3-319-66790-4_1

Cited by 147 publications

(78 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly, the top scoring system in the 2013 FER Challenge is a deep convolutional neural network [31], while the best handcrafted model ranked only in the fourth place [14]. With only a few exceptions [1,29,30], most of the recent works on facial expression recognition are based on deep learning [2,[6][7][8]11,13,16,[18][19][20][21]23,25,[34][35][36]. Some of these recent works [13,16,18,34,35] proposed to train an ensemble of convolutional neural networks for improved performance, while others [4,15] combined deep features with handcrafted features such as SIFT [22] or Histograms of Oriented Gradients (HOG) [5].…”

Section: Related Artmentioning

confidence: 99%

See 1 more Smart Citation

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Ionescu

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

In this paper, we present an approach based on convolutional neural networks (CNNs) for facial expression recognition in a difficult setting with severe occlusions. More specifically, our task is to recognize the facial expression of a person wearing a virtual reality (VR) headset which essentially occludes the upper part of the face. In order to accurately train neural networks for this setting, in which faces are severely occluded, we modify the training examples by intentionally occluding the upper half of the face. This forces the neural networks to focus on the lower part of the face and to obtain better accuracy rates than models trained on the entire faces. Our empirical results on two benchmark data sets, FER+ and AffectNet, show that our CNN models' predictions on lower-half faces are up to 13% higher than the baseline CNN models trained on entire faces, proving their suitability for the VR setting. Furthermore, our models' predictions on lower-half faces are no more than 10% under the baseline models' predictions on full faces, proving that there are enough clues in the lower part of the face to accurately predict facial expressions.

show abstract

Section: Related Artmentioning

confidence: 99%

“…In the past few years, most works [2,6,8,11,13,16,[18][19][20][21]23,25,31,[34][35][36] have focused on building and training deep neural networks in order to achieve stateof-the-art results. Engineered models based on handcrafted features [1,14,29,30] Fig.…”

Section: Introductionmentioning

confidence: 99%

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Ionescu

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…Interestingly, the top scoring system in the 2013 FER Challenge is a deep convolutional neural network [34], while the best handcrafted model ranked only on the fourth place [15]. With only a few exceptions [1,32,33], most of the recent works on facial expression recognition are based on deep learning [2,9,10,13,14,17,21,22,24,23,26,28,38,39,40]. Some of these recent works [14,17,21,38,39] proposed to train an ensemble of convolutional neural networks for improved performance, while others [6,16] combined deep features with handcrafted features such as SIFT [25] or Histograms of Oriented Gradients (HOG) [8].…”

Section: Related Workmentioning

confidence: 99%

“…In the past few years, most works [2,9,10,13,14,17,21,22,24,23,26,28,34,38,39,40] have focused on building and training deep neural networks in order to achieve state-of-the-art results. Engineered models based on handcrafted features [1,15,32,33] have drawn very little attention, since such models usually yield less accurate results compared to deep learning models.…”

Section: Introductionmentioning

confidence: 99%

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

2019

View full text Add to dashboard Cite

We present an approach that combines automatic features learned by convolutional neural networks (CNN) and handcrafted features computed by the bag-of-visual-words (BOVW) model in order to achieve state-of-the-art results in facial expression recognition. To obtain automatic features, we experiment with multiple CNN architectures, pretrained models and training procedures, e.g. Dense-Sparse-Dense. After fusing the two types of features, we employ a local learning framework to predict the class label for each test image. The local learning framework is based on three steps. First, a k-nearest neighbors model is applied in order to select the nearest training samples for an input test image. Second, a one-versus-all Support Vector Machines (SVM) classifier is trained on the selected training samples. Finally, the SVM classifier is used to predict the class label only for the test image it was trained for. Although we have used local learning in combination with handcrafted features in our previous work, to the best of our knowledge, local learning has never been employed in combination with deep features. The experiments on the 2013 Facial Expression Recognition (FER) Challenge data set, the FER+ data set and the AffectNet data set demonstrate that our approach achieves state-of-the-art results. With a top accuracy of 75.42% on FER 2013, 87.76% on the FER+, 59.58% on AffectNet 8-way classification and 63.31% on AffectNet 7-way classification, we surpass the state-of-theart methods by more than 1% on all data sets.

show abstract

“…First, by labelling the naturalistic speech segments and facial images were extremely difficult and mostly in the case of large datasets, For speech emotion corpus database [2], most datasets consists of different audio files recorded by different speakers and every emotion is labelled in a sequence of time and for facial image database (FER2013) [3], the images and labels represent the emotions of different people. Secondly, the labeled datasets can deteriorate from misplaced annotations that needs proper revision.…”

Section: Introductionmentioning

confidence: 99%

Classroom Student Emotions Classification from Facial Expressions and Speech Signals using Deep Learning

Sharma*¹,

Mansotra²

2019

IJRTE

View full text Add to dashboard Cite

Classroom environment is a competent platform for the students to learn and improve their understanding of the subject. An instructor’s primary responsibility lies in managing the students in a way they feel interested and focused during the class. With the aid of automated systems based on artificial intelligence, an instructor can get feedback on the students’ attention span in the class by monitoring their emotions using learning algorithms that can prove to be effective to improve the teaching style of the instructor that can in turn have positive effects on the class. In this paper, we propose an LSTM recurrent neural network trained on an emotional corpus database to extract the speech features and convolutional neural networks trained on the FER2013 facial emotion recognition database were used to predict the speech and facial emotions of the students respectively, in real-time. The live video and audio sequence of the students captured is fed to the learned model to classify the emotions individually. Once the emotions such as anger, sadness, happiness, surprise, fear, disgust and neutral were identified, a decision-making mechanism was used to analyze the predicted emotions and choose the overall group emotion by virtue of the highest peak value achieved. This research approach has the potential to be deployed in video conferences, online classes etc. This implementation proposal should effectively improve the classification accuracy and the relatability of the detected student emotions and facilitate in the design of sophisticated automated learning systems that can be a valuable tool in evaluating both the students and the instructors. The adapted research methodologies and their results are discussed and found to perform suggestively better than the other research works used in the comparison

show abstract

Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013

Cited by 147 publications

References 27 publications

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

Classroom Student Emotions Classification from Facial Expressions and Speech Signals using Deep Learning

Contact Info

Product

Resources

About