Modeling Emotions in Robotic Socially Believable Behaving Systems

Esposito, Anna; Jain, Lakhmi C.

doi:10.1007/978-3-319-31056-5_2

Cited by 12 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Laughter commonly occurs in daily interactions, not only simply in relation to funny situations, but also to some type of attitudes (like friendliness or interest), having an important social function in human-human communication [1,2], as well as positive influences on human's health [3,4]. Laughter also provides social signals that allow our conversations to flow smoothly among topics; to help us repairing conversations that are breaking down; and to finish our conversations on a positive note [5,6].…”

Section: Introductionmentioning

confidence: 99%

Analysis and generation of laughter motions, and evaluation in an android robot

Ishi¹,

Minato²,

Ishiguro³

2019

SIP

View full text Add to dashboard Cite

Laughter commonly occurs in daily interactions, and is not only simply related to funny situations, but also to expressing some type of attitudes, having important social functions in communication. The background of the present work is to generate natural motions in a humanoid robot, so that miscommunication might be caused if there is mismatching between audio and visual modalities, especially in laughter events. In the present work, we used a multimodal dialogue database, and analyzed facial, head, and body motion during laughing speech. Based on the analysis results of human behaviors during laughing speech, we proposed a motion generation method given the speech signal and the laughing speech intervals. Subjective experiments were conducted using our android robot by generating five different motion types, considering several modalities. Evaluation results showed the effectiveness of controlling different parts of the face, head, and upper body (eyelid narrowing, lip corner/cheek raising, eye blinking, head motion, and upper body motion control).

show abstract

Section: Introductionmentioning

confidence: 99%

Analysis and generation of laughter motions, and evaluation in an android robot

Ishi¹,

Minato²,

Ishiguro³

2019

SIP

View full text Add to dashboard Cite

show abstract

“…These data bears evidence that in the quest for implementing future socially and emotional believable human-machine interactions, personal traits, cultural specificities, and contextual instances must be accounted for [10], [18], [19]. The envisaged promising applications in this field, such as those for the early interventions and management of cognitive and physical diseases, as well as, responsive and symbiotic personal assistants simulating human perception, attention, imagination, and emotions, need a holistic perspective.…”

Section: Discussionmentioning

confidence: 87%

Differences between hearing and deaf subjects in decoding foreign emotional faces

Esposito

Cirillo

et al. 2017

2017 8th IEEE International Conference on Cognitive Infocommunications (CogInfoCom)

Self Cite

View full text Add to dashboard Cite

“…Besides, the IPC embedding can be leveraged to make inferences unrelated to engagement, based on the work done by Gurtman [24]. Therefore, this pipeline can help robots better understand the humans they interact with, which is particularly crucial in online learning platforms, chat-bots and assistive robotics [3,69], etc.…”

Section: Discussionmentioning

confidence: 99%

“…3. The successful validation of the TIB's attitude and emotion components in human-robot interactions opens up opportunities to use the pipeline for modelling other behaviours, such as inferring caution in users of socially assistive robots [3,69], measuring non-compliance in class management systems [70], and determining decisiveness in automated interviews [71].…”

Section: Future Directionsmentioning

confidence: 99%

From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions

Joshi,

Malavalli,

Rao

2023

PLoS ONE

View full text Add to dashboard Cite

Modelling the engaging behaviour of humans using multimodal data collected during human-robot interactions has attracted much research interest. Most methods that have been proposed previously predict engaging behaviour directly from multimodal features, and do not incorporate personality inferences or any theories of interpersonal behaviour in human-human interactions. This work investigates whether personality inferences and attributes from interpersonal theories of behaviour (like attitude and emotion) further augment the modelling of engaging behaviour. We present a novel pipeline to model engaging behaviour that incorporates the Big Five personality traits, the Interpersonal Circumplex (IPC), and the Triandis Theory of Interpersonal Behaviour (TIB). We extract first-person vision and physiological features from the MHHRI dataset and predict the Big Five personality traits using a Support Vector Machine. Subsequently, we empirically validate the advantage of incorporating personality in modelling engaging behaviour and present a novel method that effectively uses the IPC to obtain scores for a human’s attitude and emotion from their Big Five traits. Finally, our results demonstrate that attitude and emotion are correlates of behaviour even in human-robot interactions, as suggested by the TIB for human-human interactions. Furthermore, incorporating the IPC and the Big Five traits helps generate behavioural inferences that supplement the engaging behaviour prediction, thus enriching the pipeline. Engagement modelling has a wide range of applications in domains like online learning platforms, assistive robotics, and intelligent conversational agents. Practitioners can also use this work in cognitive modelling and psychology to find more complex and subtle relations between humans’ behaviour and personality traits, and discover new dynamics of the human psyche. The code will be made available at: https://github.com/soham-joshi/engagement-prediction-mhhri.

show abstract

Modeling Emotions in Robotic Socially Believable Behaving Systems

Cited by 12 publications

References 23 publications

Analysis and generation of laughter motions, and evaluation in an android robot

Analysis and generation of laughter motions, and evaluation in an android robot

Differences between hearing and deaf subjects in decoding foreign emotional faces

From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions

Contact Info

Product

Resources

About