Toward Unbiased Facial Expression Recognition in the Wild via Cross-Dataset Adaptation

Abstract

Despite various success in computer vision with facial images (e.g., face detection, recognition, and generation), facial expression recognition is still a challenging problem yet to be solved. This is because of simple but fundamental bottlenecks: 1) no global agreement on different facial expressions, 2) significant dataset biases that prevent cross-dataset analysis for a large-scale study, and 3) high class imbalance in in-the-wild datasets that causes inconsistency in predicting expressions in images using a machine learning algorithm. To tackle these issues, we propose a novel Deep Learning approach via adaptive cross-dataset scheme. We combine multiple in-the-wild datasets to secure sufficient training samples while minimizing dataset bias using ideas of reversal gradients to retain generality. For this, we introduce a flexible objective function that can control for skewed label distributions in the dataset. Incorporating these ideas, together with the ResNet pipeline as a backbone, we carried extensive experiments to validate our ideas using three independent in-the-wild facial expression datasets, which first confirmed bias from different datasets and yielded improved performance on facial expression recognition using the multi-site dataset.

Facial Expressions Recognition Via CNNCraft-net for Static RGB Images

“…Also, the proposed _ ℎ_ method that can load images and extract handcraft features using method on the fly during the [10] achieved top accuracy of 75.42% on FER2013 59.58% on the AffectNet (augmented and down-sampling) dataset for eight expressions and 63.31% for 7 expressions. While, Han, Byungok, et al [9] achieved accuracy for 58.89% for seven expressions in AffectNet.…”

“…Many studies used and inspired the convolution neural networks for FER problem whether with finetuning or modifying the architecture or ensembling with other architectures such as Han, Byungok, et al [9] proposed a cross-dataset adaptation method for merging different datasets to get sufficient sample size to train a deep learning model and reduce biases that exist across different datasets via proposed separate feature extractor and pseudo-label extractor.…”

Towards Sustainable and Technology-Enabled Engineering Psychology in Production

“…Constant parameters include the preexisting AMR at the laboratory facilities, with its respective mapping and routing system, the use of a pretrained algorithm due to time restrictions and the physical characteristics of the testing subjects. The combination of the latter two leads to biased results, stemming from the subjectiveness of the human emotions determined by personal and cultural differences [22] [23]. Consequently, the results might not be optimal and unbiased to the subject in front of the camera.…”