Group-level emotion recognition based on faces, scenes, skeletons features

Li, Dejian; Luo, Ruiming; Sun, Shouqian

doi:10.1117/12.2557175

Cited by 3 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [80], CNNs are used for analysis of faces, scenes, and bodies, and [81] and adds a skeleton analysis to the face and scene analysis, all done with CNNs. Faces, scenes and skeletons are also analysed with CNNs in [82], where on the face-level the CNN output is fed to an LSTM, and where on the scene-level an attention mask is placed over the image. Attention is also applied in [2], where next to faces, scenes, and skeletons, visual attentions are included (salient regions important for emotion detection, found by neural attention mechanisms) by feeding 16 salient regions to a CNN and LSTM.…”

Section: Hybrid Approachesmentioning

confidence: 99%

“…Faces, scenes, and upper bodies [34], [35] Faces, scenes, and bodies/skeletons [80], [81], [82] Faces, scenes, skeletons, [2], [42] and visual attentions/objects Faces and objects [83] Faces, scenes, and places [24] and scene analysis), or fusion of individual emotions in a bottom-up approach.…”

Section: Aspects Description Studiesmentioning

confidence: 99%

“…One or more fully-connected layers for fusion of individual aspects is employed in [1], [69], and [10]. An LSTM for feature fusion is adopted in [15] and [70], whereas [83], [82] and [35] use an SVM, the latter with Revisited Localized Multiple Kernel Learning (MKL). MKL is also proposed for fusion (and prediction) in [9].…”

Section: Fusion Of Different Analysis Aspectsmentioning

confidence: 99%

“…The five biggest faces are used, performing zero-padding if there are less than five faces detected. Other studies, making use of the sequential nature of LSTMs to take a variable number of faces, are [70], [82], [64], and [73].…”

Section: Feature-level Fusionmentioning

confidence: 99%

See 3 more Smart Citations

Automatic Emotion Recognition for Groups: A Review

Veltmeijer

Gerritsen

Hindriks

2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

This review aims to summarize and describe research on the topic of automatic group emotion recognition. In recent years, the topic of emotion analysis of groups or crowds has gained interest, with studies performing emotion detection in different contexts, using different datasets and modalities (such as images, video, audio, social media messages), and taking different approaches. Articles are included after an innovative search method, including Dense Query Extraction and automatic cross-referencing. Discussed are the types of groups and emotion models considered in automatic emotion recognition research, common datasets for all modalities, general approaches taken, and reported performances. These performances are discussed, followed by an analysis of the application possibilities of the discussed methods. To ensure clear, replicable, and comparable studies, we suggest research should test on multiple, common datasets and report on multiple metrics, when possible. Implementation details and code should be made available where possible. An area of interest for future work is to build systems with more real-world application possibilities, coping with changing group sizes, different emotional subgroups, and changing emotions over time, while having a higher robustness and working with datasets with reduced biases.

show abstract

Section: Hybrid Approachesmentioning

confidence: 99%

Section: Aspects Description Studiesmentioning

confidence: 99%

Section: Fusion Of Different Analysis Aspectsmentioning

confidence: 99%

Section: Feature-level Fusionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Emotion Recognition for Groups: A Review

Veltmeijer

Gerritsen

Hindriks

2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Some new methods were also introduced to improve the efficiency of feature fusion in GER. For example, long short-term memory (LSTM) was used to aggregate the features of scenes and faces [18][19][20]. Graph Neural Networks were also employed to fuse different emotional cues and exploit the underlying relations and interactions between the emotional cues [21].…”

Section: Group Emotion Recognitionmentioning

confidence: 99%

Semi-Supervised Group Emotion Recognition Based on Contrastive Learning

et al. 2022

View full text Add to dashboard Cite

The performance of all learning-based group emotion recognition (GER) methods depends on the number of labeled samples. Although there are lots of group emotion images available on the Internet, labeling them manually is a labor-intensive and cost-expensive process. For this reason, datasets for GER are usually small in size, which limits the performance of GER. Considering labeling manually is challenging, using limited labeled images and a large number of unlabeled images in the network training is a potential way to improve the performance of GER. In this work, we propose a semi-supervised group emotion recognition framework based on contrastive learning to learn efficient features from both labeled and unlabeled images. In the proposed method, the unlabeled images are used to pretrain the backbone by a contrastive learning method, and the labeled images are used to fine-tune the network. The unlabeled images are then given pseudo-labels by the fine-tuned network and used for further training. In order to alleviate the uncertainty of the given pseudo-labels, we propose a Weight Cross-Entropy Loss (WCE-Loss) to suppress the influence of the samples with unreliable pseudo-labels in the training process. Experiment results on three prominent benchmark datasets for GER show the effectiveness of the proposed framework and its superiority compared with other competitive state-of-the-art methods.

show abstract

Group-Level Emotion Recognition Using Hierarchical Dual-Branch Cross Transformer with Semi-Supervised Learning

Xu,

Huang

2024

2024 IEEE 4th International Conference on Software Engineering and Artificial Intelligence (SEAI)

View full text Add to dashboard Cite

Group-level emotion recognition based on faces, scenes, skeletons features

Cited by 3 publications

References 0 publications

Automatic Emotion Recognition for Groups: A Review

Automatic Emotion Recognition for Groups: A Review

Semi-Supervised Group Emotion Recognition Based on Contrastive Learning

Group-Level Emotion Recognition Using Hierarchical Dual-Branch Cross Transformer with Semi-Supervised Learning

Contact Info

Product

Resources

About