Facial Expression Recognition on Static Images

Ngo, Quan T.; Yoon, Seokhoon

doi:10.1007/978-3-030-35653-8_42

Cited by 7 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A variety of transfer learning techniques have been explored for video/image-based automatic emotion recognition ( Table 2). The state-of-art transfer learning approach to the video-based emotion recognition includes obtaining highlevel features using mainly a convolutional neural network (CNN) trained on large sources of data (e.g., VGG; Simonyan and Zisserman, 2014) (Kaya et al, 2017;Aly and Abbott, 2019;Ngo and Yoon, 2019, or transfering the knowledge from higherquality auxiliary image datasets (e.g., skeleton or color of an image, image with description text) (Xu et al, 2016). Source datasets in this case might not necessarily contain the same labeled classes as the target dataset.…”

Section: Transfer Learning For Video/image-based Emotion Recognitionmentioning

confidence: 99%

A Review of Generalizable Transfer Learning in Automatic Emotion Recognition

Feng

Chaspari

2020

Front. Comput. Sci.

View full text Add to dashboard Cite

Automatic emotion recognition is the process of identifying human emotion from signals such as facial expression, speech, and text. Collecting and labeling such signals is often tedious and many times requires expert knowledge. An effective way to address challenges related to the scarcity of data and lack of human labels, is transfer learning. In this manuscript, we will describe fundamental concepts in the field of transfer learning and review work which has successfully applied transfer learning for automatic emotion recognition. We will finally discuss promising future research directions of transfer learning for improving the generalizability of automatic emotion recognition systems.

show abstract

Section: Transfer Learning For Video/image-based Emotion Recognitionmentioning

confidence: 99%

A Review of Generalizable Transfer Learning in Automatic Emotion Recognition

Feng

Chaspari

2020

Front. Comput. Sci.

View full text Add to dashboard Cite

show abstract

“…To compare our model with existing models, we also conducted experiments in [46] where authors used a ResNet-50 model [47] which is pre-trained with ImageNet data [41] for object detection task as their base model and then fine-tuned it with AffectNet data for FER task. It is worthwhile to note that, in [46], authors fine-tuned their model using only the conventional softmax loss.…”

Section: Resultsmentioning

confidence: 99%

“…It is worthwhile to note that in preliminary experiments, Ngo and Yoon showed the improvement in recognition performance when using ImageNet data [41] as auxiliary data for building a transfer learning-based FER model [46]. The authors fine-tuned a ResNet-50 [47] model, which was pre-trained with ImageNet data.…”

Section: Transfer Learning For Facial Expression Recognitionmentioning

confidence: 99%

“…This may help improve the performance of the transfer learning-based FER system. Furthermore, in [46], authors simply fine-tuned their models using softmax loss and did not consider the imbalanced data problem, which degraded the recognition performance of FER system. In this paper, we focus on handling the shortage of data as well as tackling the imbalanced data problem by applying weighted loss and auxiliary loss approaches.…”

Section: Transfer Learning For Facial Expression Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Facial Expression Recognition Based on Weighted-Cluster Loss and Deep Transfer Learning Using a Highly Imbalanced Dataset

Ngo

Yoon

2020

Sensors

Self Cite

View full text Add to dashboard Cite

Facial expression recognition (FER) is a challenging problem in the fields of pattern recognition and computer vision. The recent success of convolutional neural networks (CNNs) in object detection and object segmentation tasks has shown promise in building an automatic deep CNN-based FER model. However, in real-world scenarios, performance degrades dramatically owing to the great diversity of factors unrelated to facial expressions, and due to a lack of training data and an intrinsic imbalance in the existing facial emotion datasets. To tackle these problems, this paper not only applies deep transfer learning techniques, but also proposes a novel loss function called weighted-cluster loss, which is used during the fine-tuning phase. Specifically, the weighted-cluster loss function simultaneously improves the intra-class compactness and the inter-class separability by learning a class center for each emotion class. It also takes the imbalance in a facial expression dataset into account by giving each emotion class a weight based on its proportion of the total number of images. In addition, a recent, successful deep CNN architecture, pre-trained in the task of face identification with the VGGFace2 database from the Visual Geometry Group at Oxford University, is employed and fine-tuned using the proposed loss function to recognize eight basic facial emotions from the AffectNet database of facial expression, valence, and arousal computing in the wild. Experiments on an AffectNet real-world facial dataset demonstrate that our method outperforms the baseline CNN models that use either weighted-softmax loss or center loss.

show abstract

“…To approach transfer learning on emotion recognition, there is a requirement to obtain high-level features using CNN, which is trained on huge datasets (e.g., [187][188][189][190]). The originally trained datasets might not necessarily contain the same labeled classes compared to target classes on the target dataset, different from the initial model trained upon.…”

Section: Transfer Learning In Emotion Detectionmentioning

confidence: 99%

A Survey of AI-Based Facial Emotion Recognition: Features, ML & DL Techniques, Age-Wise Datasets and Future Directions

et al. 2021

View full text Add to dashboard Cite

Facial expressions are mirrors of human thoughts and feelings. It provides a wealth of social cues to the viewer, including the focus of attention, intention, motivation, and emotion. It is regarded as a potent tool of silent communication. Analysis of these expressions gives a significantly more profound insight into human behavior. AI-based Facial Expression Recognition (FER) has become one of the crucial research topics in recent years, with applications in dynamic analysis, pattern recognition, interpersonal interaction, mental health monitoring, and many more. However, with the global push towards online platforms due to the Covid-19 pandemic, there has been a pressing need to innovate and offer a new FER analysis framework with the increasing visual data generated by videos and photographs.Furthermore, the emotion-wise facial expressions of kids, adults, and senior citizens vary, which must also be considered in the FER research. Lots of research work has been done in this area. However, it lacks a comprehensive overview of the literature that showcases the past work done and provides the aligned future directions. In this paper, the authors have provided a comprehensive evaluation of AI-based FER methodologies, including datasets, feature extraction techniques, algorithms, and the recent breakthroughs with their applications in facial expression identification. To the best of the author's knowledge, this is the only review paper stating all aspects of FER for various age brackets and would significantly impact the research community in the coming years.

show abstract

Facial Expression Recognition on Static Images

Cited by 7 publications

References 8 publications

A Review of Generalizable Transfer Learning in Automatic Emotion Recognition

A Review of Generalizable Transfer Learning in Automatic Emotion Recognition

Facial Expression Recognition Based on Weighted-Cluster Loss and Deep Transfer Learning Using a Highly Imbalanced Dataset

A Survey of AI-Based Facial Emotion Recognition: Features, ML & DL Techniques, Age-Wise Datasets and Future Directions

Contact Info

Product

Resources

About