Relevance-Based Data Masking: A Model-Agnostic Transfer Learning Approach for Facial Expression Recognition

Abstract

Deep learning approaches are now a popular choice in the field of automatic emotion recognition (AER) across various modalities. Due to the high costs of manually labeling human emotions however, the amount of available training data is relatively scarce in comparison to other tasks. To facilitate the learning process and reduce the necessary amount of training-data, modern approaches therefore often rely on leveraging knowledge from models that have already been trained on related tasks where data is available abundantly. In this work we introduce a novel approach to transfer learning, which addresses two shortcomings of traditional methods: The (partial) inheritance of the original models structure and the restriction to other neural network models as an input source. To this end we identify the parts in the input that have been relevant for the decision of the model we want to transfer knowledge from, and directly encode those relevant regions in the data on which we train our new model. To validate our approach we performed experiments on well-established datasets for the task of automatic facial expression recognition. The results of those experiments are suggesting that our approach helps to accelerate the learning process.

Comparing Facial Expression Recognition in Humans and Machines: Using CAM, GradCAM, and Extremal Perturbation

“…However, by training the binary classifiers until they reach training accuracy of 90% and combining the classification results by weighted vote method, we obtained a model that is less biased towards major expressions. Interestingly, we failed to improve the model using attentional information from humans as using the masked images as fine-tuning dataset proved an inadequate method for guiding the attention of a CNN-based model -this is in some way similar to the work by [26] who also found little to no improvement when using masked images. In the future, we will work on incorporating the attention mechanism to our model to channel the model's attention to meaningful regions more effectively.…”

“…One key aspect of Knowledge Distillation is that it can transfer knowledge across models with different structure. It even enables human-machine transfer learning: [26] implemented this type of learning, albeit indirectly. The researchers first trained a Teacher model on an FER dataset and obtained a saliency map for each image by visualizing the activations of the Teacher model.…”

Study Of Deep Learning Techniques For Differently Abled Applications And Detection Techniques

“…Schiller et al, [12] proposed a relevance based data masking for facial expression recognition to learn automatic emotion recognition. The proposed method can overcome the issues of the traditional FR systems which includes inheritance of the original model structure and restriction to the neural network structure.…”

Researching a machine learning algorithm for a face recognition system

“…In [1] propose a new way to transform learning into automatic emotion recognition (AER) in different ways. An estimation of the model showed that the new model could fit in more quickly to a new place when it was necessary to keep attention on parts of the input that were mature pertinent [2] automated method of face recognition using a convolutional neural network (CNN) by learning transfer approach.…”