Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

Wang, Kai; Peng, Xiaojiang; Yang, Jianfei; Meng, Debin; Qiao, Yu

doi:10.1109/tip.2019.2956143

Cited by 653 publications

(391 citation statements)

References 55 publications

Supporting

Mentioning

391

Contrasting

Order By: Relevance

“…Our approach produces significantly better results than the recent studies on both metrics. For WA, we get 88.98%, which is improved by more than 2% in absolute terms or 2.4% relatively, compared to Wang et al [39]. In the UA metric, our approach is 4.05% better in absolute terms compared to [46] or 5.28% relatively.…”

Section: B Experimental Resultsmentioning

confidence: 56%

“…Table 4 gives the results for the RAF-DB dataset. In previous studies, the methods in [38], [39], [45] report results in WA metric, and others [46], [47] report UA metric. We report and compare with previous findings in both WA and UA metrics.…”

Section: B Experimental Resultsmentioning

confidence: 99%

“…The test set has not yet been published, so most previous studies used validation set as the test set [13], [37]- [40]. Because the contempt emotion is rare in the natural world, some studies [40] used only seven emotions while other studies [13], [38], [39] analyzed all eight emotions. Another study used both eight and seven expressions [37].…”

Section: Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition

et al. 2020

View full text Add to dashboard Cite

Facial Expression Recognition (FER) is a challenging task that improves natural humancomputer interaction. This paper focuses on automatic FER on a single in-the-wild (ITW) image. ITW images suffer real problems of pose, direction, and input resolution. In this study, we propose a pyramid with super-resolution (PSR) network architecture to solve the ITW FER task. We also introduce a prior distribution label smoothing (PDLS) loss function that applies the additional prior knowledge of the confusion about each expression in the FER task. Experiments on the three most popular ITW FER datasets showed that our approach outperforms all the state-of-the-art methods.

show abstract

Section: B Experimental Resultsmentioning

confidence: 56%

Section: B Experimental Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The accuracy of facial expression is affected by many factors, among them occlusion (resulted from some shelfs, for example, wearing spectacles) is one of the most prominent, and it is a hot research topic. Many methods are put forward to deal with such a problem, and many progresses have been achieved [59][60][61].…”

Section: Discussionmentioning

confidence: 99%

Ground Truth Dataset for EEG-Based Emotion Recognition With Visual Indication

et al. 2020

View full text Add to dashboard Cite

Along with extensive and successive applications, emotion recognition based on electroencephalogram has attracted more and more researchers. However, how to acquire sufficient and high-quality real emotion electroencephalogram to train emotion recognition model has always been a bottleneck issue in the electroencephalogram-based emotion recognition research field. Because the subject's emotion is easy to be affected by many factors and subjects do not always evoke emotion well, it's very hard to determine whether the real emotion electroencephalogram appears or not in the experiments. On the contrary to electroencephalogram, facial expression obtained without any intentional control is easy to be recognized by computers, and is also one of the primary and reliable cues for understanding emotions. Inspired by such a common sense, we proposed an approach to building up ground truth electroencephalogram dataset with visual indication. Firstly, the relationship between facial expression and electroencephalogram is analyzed in details from the viewpoints of biophysics and correlation. Secondly, based on the analysis result that when the subject's facial expression is evoked, it should be accompanied by the emotional electroencephalogram corresponding to it, the method of building up the ground truth electroencephalogram dataset with visual indication is put forward along with the computer automatic implementation. Thirdly, we have built up a ground truth electroencephalogram dataset which covers 3 kinds of emotions (i.e. joy, sadness, and neutral) of the subjects who are undergraduate and graduate students from Minzu University of China. Lastly, the validation of the established dataset is tested by the comparative experiments between the long short term memory emotion recognition models trained with two different datasets respectively. One dataset contains both truth and false electroencephalograms and another one only contains the truth data.

show abstract

“…Many previous studies on automatic engagement recognition are focused on the perceived engagement (i.e., engagement received by an external people) [9]. Various automatic engagement prediction systems have been proposed using multi-modal information such as student response [11], facial [13,17,19,20] or body movements in learning videos [5,23], behavior in test quizzes [10] and even advanced physiological and neurological measures [8]. Among them, the video data is a good trade-off between capturing convenience and granularity.…”

Section: Related Workmentioning

confidence: 99%

Deep Recurrent Multi-instance Learning with Spatio-temporal Features for Engagement Intensity Prediction

Yang

Wang

Peng

et al. 2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

This paper presents our approach for the engagement intensity regression task of EmotiW 2019. The task is to predict the engagement intensity value of a student when he or she is watching an online MOOCs video in various conditions. Based on our winner solution last year, we mainly explore head features and body features with a bootstrap strategy and two novel loss functions in this paper. We maintain the framework of multi-instance learning with long short-term memory (LSTM) network, and make three contributions. First, besides of the gaze and head pose features, we explore facial landmark features in our framework. Second, inspired by the fact that engagement intensity can be ranked in values, we design a rank loss as a regularization which enforces a distance margin between the features of distant category pairs and adjacent category pairs. Third, we use the classical bootstrap aggregation method to perform model ensemble which randomly samples a certain training data by several times and then averages the model predictions. We evaluate the performance of our method and discuss the influence of each part on the validation dataset. Our methods finally win 3rd place with MSE of 0.0626 on the testing set.

show abstract

Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

Cited by 653 publications

References 55 publications

Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition

Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition

Ground Truth Dataset for EEG-Based Emotion Recognition With Visual Indication

Deep Recurrent Multi-instance Learning with Spatio-temporal Features for Engagement Intensity Prediction

Contact Info

Product

Resources

About