This review aims to summarize and describe research on the topic of automatic group emotion recognition. In recent years, the topic of emotion analysis of groups or crowds has gained interest, with studies performing emotion detection in different contexts, using different datasets and modalities (such as images, video, audio, social media messages), and taking different approaches. Articles are included after an innovative search method, including Dense Query Extraction and automatic cross-referencing. Discussed are the types of groups and emotion models considered in automatic emotion recognition research, common datasets for all modalities, general approaches taken, and reported performances. These performances are discussed, followed by an analysis of the application possibilities of the discussed methods. To ensure clear, replicable, and comparable studies, we suggest research should test on multiple, common datasets and report on multiple metrics, when possible. Implementation details and code should be made available where possible. An area of interest for future work is to build systems with more real-world application possibilities, coping with changing group sizes, different emotional subgroups, and changing emotions over time, while having a higher robustness and working with datasets with reduced biases.

Both social group detection and group emotion recognition in images are growing fields of interest, but never before have they been combined. In this work we aim to detect emotional subgroups in images, which can be of great importance for crowd surveillance or event analysis. To this end, human annotators are instructed to label a set of 171 images, and their recognition strategies are analysed. Three main strategies for labeling images are identified, with each strategy assigning either 1) more weight to emotions (emotion-based fusion), 2) more weight to spatial structures (group-based fusion), or 3) equal weight to both (summation strategy). Based on these strategies, algorithms are developed to automatically recognize emotional subgroups. In particular, K-means and hierarchical clustering are used with location and emotion features derived from a fine-tuned VGG network. Additionally, we experiment with face size and gaze direction as extra input features. The best performance comes from hierarchical clustering with emotion, location and gaze direction as input.

In recent years, many agent-based models of human groups have implemented a mechanism of emotion contagion, yet empirical validation is lagging behind. The aim of the present paper is to validate an agent-based model of emotion contagion at the level of group emotion, by comparing simulations against the emotional development of real people in small groups. To study the effect of emotion contagion, the participants interacted via a video call, where they were virtually placed in different social environments while they played a quiz. This allowed the exchange of emotion among all, some or none of the participants. The patterns of emotional development in the empirical results supported our hypotheses based on literature of emotion contagion and social norms. Further, the simulations with the complete model resembled many of these patterns. When emotion contagion was disabled in the model, the resemblance decreased. These results give a first indication that emotion contagion occurs in groups that meet via video calls, and can in-part be predicted by the proposed model of emotion contagion. Yet, further study with a larger and more diverse empirical sample is needed, as well as comparisons across contagion mechanisms, to draw stronger conclusions and ultimately justify societal application.

No abstract