From individual to group-level emotion recognition: EmotiW 5.0

Dhall, Abhinav; Goecke, Roland; Ghosh, Shreya; Joshi, Jyoti; Hoey, Jesse; Gedeon, Tom

doi:10.1145/3136755.3143004

Cited by 180 publications

(130 citation statements)

References 21 publications

Supporting

Mentioning

130

Contrasting

Order By: Relevance

“…Group-level affect recognition is a subdiscipline of FER, where the goal is to assess the overall expression of all persons in an image. It has been featured in the EmotiW competition in 2016 [12] and 2017 [86]. For this purpose, spatial features are typically extracted for each person, and fused in some way-e.g., by considering multiple faces as a sequence and applying an LSTM [15].…”

Section: Learning Spatial Features For Fermentioning

confidence: 99%

Deep Learning for Human Affect Recognition: Insights and New Developments

Rouast

Adam

Chiong

2021

IEEE Trans. Affective Comput.

160

View full text Add to dashboard Cite

Automatic human affect recognition is a key step towards more natural human-computer interaction. Recent trends include recognition in the wild using a fusion of audiovisual and physiological sensors, a challenging setting for conventional machine learning algorithms. Since 2010, novel deep learning algorithms have been applied increasingly in this field. In this paper, we review the literature on human affect recognition between 2010 and 2017, with a special focus on approaches using deep neural networks. By classifying a total of 950 studies according to their usage of shallow or deep architectures, we are able to show a trend towards deep learning. Reviewing a subset of 233 studies that employ deep neural networks, we comprehensively quantify their applications in this field. We find that deep learning is used for learning of (i) spatial feature representations, (ii) temporal feature representations, and (iii) joint feature representations for multimodal sensor data. Exemplary state-of-the-art architectures illustrate the progress. Our findings show the role deep architectures will play in human affect recognition, and can serve as a reference point for researchers working on related applications.

show abstract

Section: Learning Spatial Features For Fermentioning

confidence: 99%

Deep Learning for Human Affect Recognition: Insights and New Developments

Rouast

Adam

Chiong

2021

IEEE Trans. Affective Comput.

160

View full text Add to dashboard Cite

show abstract

“…The inception V3 network is similar to the original work [30], which was proposed for the classification on the ImageNet task GAF In the word cloud of the survey result, people mentioned some group-level emotion-related keywords such as 'violence', 'happy', 'angry', 'upset' etc. Thus, we perform experiments with joint training for GCS and group emotion (three classes positive, neutral and negative [13]). The motivation is to explore the usefulness of GCS of a group as an attribute for group emotion prediction.…”

Section: A Image-level Analysismentioning

confidence: 99%

“…1) To the best of our knowledge, this is the first study proposing AGC prediction in images; 2) We compare two cohesion models, representing scene (holistic) and face-level information respectively, and show that the former contributes more to the perception of cohesion; 3) We label and extend the Group Affect Database [13] with group cohesion labels and propose the GAF Cohesion database (sample images from the database are shown in Fig. 1); 4) From our experimental results, we observed that the perceived group emotion is related to group cohesiveness (Section VI).…”

Section: Introductionmentioning

confidence: 99%

Predicting Group Cohesiveness in Images

Ghosh

Dhall

Sebe

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

The cohesiveness of a group is an essential indicator of the emotional state, structure and success of a group of people. We study the factors that influence the perception of grouplevel cohesion and propose methods for estimating the humanperceived cohesion on the group cohesiveness scale. In order to identify the visual cues (attributes) for cohesion, we conducted a user survey. Image analysis is performed at a group-level via a multi-task convolutional neural network. For analyzing the contribution of facial expressions of the group members for predicting the Group Cohesion Score (GCS), a capsule network is explored. We add GCS to the Group Affect database and propose the 'GAF-Cohesion database'. The proposed model performs well on the database and is able to achieve near human-level performance in predicting a group's cohesion score. It is interesting to note that group cohesion as an attribute, when jointly trained for group-level emotion prediction, helps in increasing the performance for the later task. This suggests that group-level emotion and cohesion are correlated.

show abstract

“…Expression datasets: Several facial expression datasets have been created in the past that consist of face images labeled with discrete emotion categories [4,9,10,11,16,17,31,34,40,41,43,54,55], facial action units [4,34,36,37,43], and strengths of valence and arousal [25,27,28,40,44]. While these datasets played a significant role in the advancement of automatic facial expression analysis in terms of emotion recognition, action unit detection and valence-arousal estimation, they are not the best fit for learning a compact expression embedding space that mimics human visual preferences.…”

Section: Related Workmentioning

confidence: 99%

A Compact Embedding for Facial Expression Similarity

Vemulapalli

Agarwala

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Most of the existing work on automatic facial expression analysis focuses on discrete emotion recognition, or facial action unit detection. However, facial expressions do not always fall neatly into pre-defined semantic categories. Also, the similarity between expressions measured in the action unit space need not correspond to how humans perceive expression similarity. Different from previous work, our goal is to describe facial expressions in a continuous fashion using a compact embedding space that mimics human visual preferences. To achieve this goal, we collect a large-scale faces-in-the-wild dataset with human annotations in the form: Expressions A and B are visually more similar when compared to expression C, and use this dataset to train a neural network that produces a compact (16-dimensional) expression embedding. We experimentally demonstrate that the learned embedding can be successfully used for various applications such as expression retrieval, photo album summarization, and emotion recognition. We also show that the embedding learned using the proposed dataset performs better than several other embeddings learned using existing emotion or action unit datasets.

show abstract

From individual to group-level emotion recognition: EmotiW 5.0

Cited by 180 publications

References 21 publications

Deep Learning for Human Affect Recognition: Insights and New Developments

Deep Learning for Human Affect Recognition: Insights and New Developments

Predicting Group Cohesiveness in Images

A Compact Embedding for Facial Expression Similarity

Contact Info

Product

Resources

About