Towards Facial Expression Robustness in Multi-scale Wild Environments

Freire-Obregón, David; Hernández-Sosa, Daniel; Santana, Oliverio J.; Lorenzo-Navarro, Javier; Castrillón-Santana, Modesto

doi:10.1007/978-3-031-43148-7_16

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Other1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Multimodal emotion recognition based on a fusion of audiovisual information with temporal dynamics

Salas-Cáceres,

Lorenzo-Navarro,

Freire-Obregón

et al. 2024

Multimed Tools Appl

View full text Add to dashboard Cite

In the Human-Machine Interactions (HMI) landscape, understanding user emotions is pivotal for elevating user experiences. This paper explores Facial Expression Recognition (FER) within HMI, employing a distinctive multimodal approach that integrates visual and auditory information. Recognizing the dynamic nature of HMI, where situations evolve, this study emphasizes continuous emotion analysis. This work assesses various fusion strategies that involve the addition to the main network of different architectures, such as autoencoders (AE) or an Embracement module, to combine the information of multiple biometric cues. In addition to the multimodal approach, this paper introduces a new architecture that prioritizes temporal dynamics by incorporating Long Short-Term Memory (LSTM) networks. The final proposal, which integrates different multimodal approaches with the temporal focus capabilities of the LSTM architecture, was tested across three public datasets: RAVDESS, SAVEE, and CREMA-D. It showcased state-of-the-art accuracy of 88.11%, 86.75%, and 80.27%, respectively, and outperformed other existing approaches.

show abstract