Predicting Social Dynamics in Child-Robot Interactions with Facial Action Units

Eijndhoven, Kyana van; Wiltshire, Travis J.; Vogt, Paul

doi:10.1145/3371382.3378366

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Affective state and emotional facial expression recognition are also vital to foster HRI toward trust (Gaudiello et al, 2016 ), engagement (Kompatsiari et al, 2017 ; Babel et al, 2021 ), and rapport (Hoffmann et al, 2020 ), targeting long-term relationships between a human and a robot. The most prominent features in HRI for detecting affective states or emotions in facial expressions are arousal and valence (McColl et al, 2016 ), and so-called action units (Liu and Kappas, 2018 ; van Eijndhoven et al, 2020 ).…”

Section: Motivation and Related Workmentioning

confidence: 99%

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

et al. 2022

View full text Add to dashboard Cite

A key goal in human-robot interaction (HRI) is to design scenarios between humanoid robots and humans such that the interaction is perceived as collaborative and natural, yet safe and comfortable for the human. Human skills like verbal and non-verbal communication are essential elements as humans tend to attribute social behaviors to robots. However, aspects like the uncanny valley and different technical affinity levels can impede the success of HRI scenarios, which has consequences on the establishment of long-term interaction qualities like trust and rapport. In the present study, we investigate the impact of a humanoid robot on human emotional responses during the performance of a cognitively demanding task. We set up three different conditions for the robot with increasing levels of social cue expressions in a between-group study design. For the analysis of emotions, we consider the eye gaze behavior, arousal-valence for affective states, and the detection of action units. Our analysis reveals that the participants display a high tendency toward positive emotions in presence of a robot with clear social skills compared to other conditions, where we show how emotions occur only at task onset. Our study also shows how different expression levels influence the analysis of the robots' role in HRI. Finally, we critically discuss the current trend of automatized emotion or affective state recognition in HRI and demonstrate issues that have direct consequences on the interpretation and, therefore, claims about human emotions in HRI studies.

show abstract

Section: Motivation and Related Workmentioning

confidence: 99%

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

et al. 2022

View full text Add to dashboard Cite

show abstract

“…AU detection involves determining the occurrences of different AUs in a given face image [4], [5], with numerous potential applications, including in the development of interactive robotic systems and other real-world scenarios [6]- [8]. In recent years, Deep Neural Networks (DNNs) have been commonly used for detecting and localizing AUs in facial images [9]- [11].…”

Section: Introductionmentioning

confidence: 99%

Attention-Enabled Lightweight Neural Network Architecture for Detection of Action Unit Activation

Deramgozin,

Jovanovic,

Arevalillo-Herráez

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Facial Action Unit (AU) detection is of major importance in a broad range of artificial intelligence applications such as healthcare, Facial Expression Recognition (FER), and mental state analysis. In this paper, we present an innovative, resource-efficient facial AU detection model, embedding both spatial and channel attention mechanisms into a convolutional neural network (CNN). Along with a unique data input system leveraging image data and binary-encoded AU activation labels, our model enhances AU detection capabilities while simultaneously offering interpretability for FER systems. In contrast to existing state-of-the-art models, our proposal's streamlined architecture, combined with superior performance, establishes it as an ideal solution for resource-limited environments like mobile and embedded systems with computational constraints. The model was trained and evaluated utilizing the BP4D, CK+, DISFA, FER2013+, and RAF-DB datasets, with the latter two being particularly significant as they represent wild datasets for facial expression recognition. These datasets encompass ground truth emotions matched with corresponding AU activations according to the Facial Action Coding System. Various metrics, including F1 score, accuracy, and Euclidean distance, showcase its effectiveness in AU detection and interpretability.

show abstract

Predicting Social Dynamics in Child-Robot Interactions with Facial Action Units

Cited by 2 publications

References 14 publications

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Attention-Enabled Lightweight Neural Network Architecture for Detection of Action Unit Activation

Contact Info

Product

Resources

About