Interpretability Study of Transfer Learning for Emotion Classification

Abstract

Transfer learning is a powerful technique for improving the performance of deep neural networks on a target task with limited labeled data. However, it is not always clear why pre-trained models on unrelated tasks, such as ImageNet, can be effective for emotion recognition tasks. This paper evaluates the performance of transfer learning for facial expression recognition tasks and investigates the influence of network depth on performance using three different typical pre-trained models: VGG16, MobileNet, and ResNet50. The layers of the pre-trained models are frozen, and their output layer is substituted with a new output layer that corresponds to the seven-class emotion classes in the target dataset. The study demonstrates that transfer learning with pre-trained models, even though with limited labelled dataset regarding facial expressions, can be an effective approach for improving the performance of deep neural networks on emotion recognition tasks. The findings provide insights into the interpretability of transfer learning for facial expression recognition tasks and highlight the importance of fine-tuning pre-trained models for optimal performance. Additionally, the study also offers ideas for future model design in facial expression recognition tasks.