Group Emotion Recognition Based on Global and Local Features

Abstract

In order to improve the accuracy of group emotion recognition, a group emotion recognition model based on global scene feature and local face feature is constructed in this paper. When extracting global scene features, with the consideration of that the different size of the background objects may have different influences on the emotion recognition, the paper proposes a feature extraction algorithm for the global scene based on the fusion of multi-scale feature maps. With the consideration of the emotion propagation between different figures in the image, the paper proposes a LSTM based algorithm for fusion the face features among different figures. Experiments show that the global scene feature extraction algorithm proposed in this paper has higher accuracy than the global scene feature extraction algorithm based on standard network architecture. Besides, the facial emotion feature fusion algorithm based on LSTM has higher classification accuracy than the fusion algorithm based on average calculation and the algorithm based on voting. Besides, the group emotion recognition model proposed in this paper has an accuracy 24.38% higher than the benchmark method, 13.32% higher than the deep learning method and 14.57% higher than the deep learning method.INDEX TERMS Deep two-way long-term and short-term memory network, feature fusion, global scene features, group emotion recognition.

Machine Learning Based Analysis of Finnish World War II Photographers

“…The photographer analysis could be further enhanced, for example, by considering photographer intentions [10] and their photo quality [52], besides, object detection performance can be enhanced by considering information fusion approaches [6], as well as improving detection of smaller sized objects [4]. Here, we only considered person detection, while person segmentation [22], face detection with facial expression analysis [54], group-level emotion recognition [60], or age estimation [1] will open more interesting opportunities. Besides objectlevel analysis, scene recognition [63] will help to further characterize photographers.…”

Group Emotion Detection Based on Social Robot Perception

“…Similarly, in [ 27 ], several CNNs with different depths were used, but in this case using the softmax angular loss (A-Softmax) to make the learned characteristics more discriminative. In [ 40 ], after detecting the faces, the neural networks VGG-16 and MobileNet-v1 were used to extract the characteristics of each face. Instead of training a neural network, in [ 21 ], EmoNet was proposed.…”

“…More sophisticated fusion methods can also be used, such as neural networks. The Long Short-Term Memory (LSTM) neural network was used in [ 35 , 36 , 40 ] to learn how individual emotions affect the group emotion. Residual networks, such as cascade attention networks, were used in [ 25 , 32 ], to determine the influence of each face in the detection of the emotion of the group.…”

Facial Beauty Prediction via Local Feature Fusion and Broad Learning System

“…Local feature fusion [25] is a classic computer vision concept, which has been widely used in biometrics identification [26]. The traditional method is to use single local feature to train neural network to realize biometric recognition.…”

“…Min et al [28] verified the effectiveness of the multi-feature fusion algorithm in enhancing the model performance in the details of the VR panoramic image more than the single-feature fusion algorithm. Yu et al [29] suggested an emotion recognition model algorithm focusing on long short-term memory (LSTM) with the fusion of different local facial features, which acquired a 24.38% higher accuracy compared with the typical method.…”