Affect-Driven Learning of Robot Behaviour for Collaborative Human-Robot Interactions

Abstract

Collaborative interactions require social robots to share the users’ perspective on the interactions and adapt to the dynamics of their affective behaviour. Yet, current approaches for affective behaviour generation in robots focus on instantaneous perception to generate a one-to-one mapping between observed human expressions and static robot actions. In this paper, we propose a novel framework for affect-driven behaviour generation in social robots. The framework consists of (i) a hybrid neural model for evaluating facial expressions and speech of the users, forming intrinsic affective representations in the robot, (ii) an Affective Core, that employs self-organising neural models to embed behavioural traits like patience and emotional actuation that modulate the robot’s affective appraisal, and (iii) a Reinforcement Learning model that uses the robot’s appraisal to learn interaction behaviour. We investigate the effect of modelling different affective core dispositions on the affective appraisal and use this affective appraisal as the motivation to generate robot behaviours. For evaluation, we conduct a user study (n = 31) where the NICO robot acts as a proposer in the Ultimatum Game. The effect of the robot’s affective core on its negotiation strategy is witnessed by participants, who rank a patient robot with high emotional actuation higher on persistence, while an impatient robot with low emotional actuation is rated higher on its generosity and altruistic behaviour.

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

“…Therefore, the mere presence of a socially-behaving robot is not the only essential component in the interaction but also the ability of a robot to adapt to individual human personalities. Our analyses in this study fit squarely into recent research on interaction with social robots which can adapt to a human's personality (de Graaf and Allouch, 2014 ; Tanevska et al, 2019 ) and we opt for a better understanding of a robots' personality in the future as suggested by Churamani et al ( 2020 ) in everyday life such as in assistive tasks in domestic environments, driving or tutoring.…”

Towards a Personality AI for Robots: Potential Colony Capacity of a Goal-Shaped Generative Personality Model When Used for Expressing Personalities via Non-Verbal Behaviour of Humanoid Robots

“…Better understanding of personality traits and inter- and intra-individual differences in emotional expressions via speech may be necessary to achieve the integration. Churamani et al (2022) proposed a framework for constructing models capable of generating affect-modulated behaviour based on the appraisal of users’ affective states on the arousal and valence dimensions, which is in a sense generating behaviour with intra-individual differences, considering that users’ affective states are one of the most important contexts, among a few others, in expressing or understanding robot personalities. And by configuring it to make the robot behave either patiently or impatiently, or to be either excitatory or inhibitory in generating the robot’s mood, the model can generate behaviour with inter-individual differences for at least two distinct individuals on each of the three broad traits (being generous, persistent, and altruistic) that the authors confirmed to be consequential in the kind of human-robot collaboration they studied.…”

Augmenting ECG Data with Multiple Filters for a Better Emotion Recognition System

“…The ERS healthcare applications have been summarised in [ 9 ]. In [ 10 ], a multimodal ERS using facial and voice recognition is proposed to improve human–robot communication by recognizing the human’s emotion and generating an appropriate affect response.…”