Computational Emotion Analysis From Images: Recent Advances and Future Directions

Abstract

Emotions are usually evoked in humans by images. Recently, extensive research efforts have been dedicated to understanding the emotions of images. In this chapter, we aim to introduce image emotion analysis (IEA) from a computational perspective with the focus on summarizing recent advances and suggesting future directions. We begin with commonly used emotion representation models from psychology. We then define the key computational problems that the researchers have been trying to solve and provide supervised frameworks that are generally used for different IEA tasks. After the introduction of major challenges in IEA, we present some representative methods on emotion feature extraction, supervised classifier learning, and domain adaptation. Furthermore, we introduce available datasets for evaluation and summarize some main results. Finally, we discuss some open questions and future directions that researchers can pursue.

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

“…It is no doubt that facial analytics in images and videos, e.g., age, gender, ethnicity and emotion prediction [1,2], is one of the most widely studied tasks in image recognition. Thousands of papers appear every year to present all the more difficult techniques and models based on deep convolutional neural network (CNN) [3]. Ensembles of complex models won the prestigious challenges and contests [4,5].…”

“…2 , FaceNet3 , ResNet-50 model from InsightFace and Deep expectation (DEX) VGG16 networks trained on the IMDB-Wiki[14]. In contrast to our approach, all these CNNs have been fine-tuned only on specific datasets with age and gender labels, i.e., they are do not use large-scale face recognition datasets for pre-training.…”