Towards Context-Aware Facial Emotion Reaction Database for Dyadic Interaction Settings

Abstract

Emotion recognition is a significant issue in many sectors that use human emotion reactions as communication for marketing, technological equipment, or human–robot interaction. The realistic facial behavior of social robots and artificial agents is still a challenge, limiting their emotional credibility in dyadic face-to-face situations with humans. One obstacle is the lack of appropriate training data on how humans typically interact in such settings. This article focused on collecting the facial behavior of 60 participants to create a new type of dyadic emotion reaction database. For this purpose, we propose a methodology that automatically captures the facial expressions of participants via webcam while they are engaged with other people (facial videos) in emotionally primed contexts. The data were then analyzed using three different Facial Expression Analysis (FEA) tools: iMotions, the Mini-Xception model, and the Py-Feat FEA toolkit. Although the emotion reactions were reported as genuine, the comparative analysis between the aforementioned models could not agree with a single emotion reaction prediction. Based on this result, a more-robust and -effective model for emotion reaction prediction is needed. The relevance of this work for human–computer interaction studies lies in its novel approach to developing adaptive behaviors for synthetic human-like beings (virtual or robotic), allowing them to simulate human facial interaction behavior in contextually varying dyadic situations with humans. This article should be useful for researchers using human emotion analysis while deciding on a suitable methodology to collect facial expression reactions in a dyadic setting.

Augmented Cognition: Life as we don't know it

“…Researchers in the field of HRI often use insights from cognitive psychology, neuroscience, and social psychology to inform their work on designing robots that can effectively interact with humans. This includes research on how humans process nonverbal cues such as facial expressions, gestures, and body language, which are then incorporated into the design of robots to make them more human-like and to improve their ability to communicate effectively with humans (Saunderson and Nejat, 2019;Chen et al, 2023;Sham et al, 2023). On the other hand, HRI is also shaped by the values, goals, and aspirations of human culture.…”

Computer Vision in Human Analysis: From Face and Body to Clothes

“…Another topic investigated in this Special Issue is human face analysis. In particular, the published papers address different topics, including the problem of machine interaction using voice commands and facial movements [ 6 ], 3D face and body geometry reconstruction [ 7 , 8 ], and dyadic interaction analysis based on facial expressions [ 9 ]. Focusing on eye images, Gibertoni et al [ 10 ] propose a system to automatically classify the eye status in images acquired through an ophthalmic tool.…”