Learning Socially Appropriate Robot Approaching Behavior Toward Groups using Deep Reinforcement Learning

Abstract

Deep reinforcement learning has recently been widely applied in robotics to study tasks such as locomotion and grasping, but its application to social human-robot interaction (HRI) remains a challenge. In this paper, we present a deep learning scheme that acquires a prior model of robot approaching behavior in simulation and applies it to real-world interaction with a physical robot approaching groups of humans. The scheme, which we refer to as Staged Social Behavior Learning (SSBL), considers different stages of learning in social scenarios. We learn robot approaching behaviors towards small groups in simulation and evaluate the performance of the model using objective and subjective measures in a perceptual study and a HRI user study with human participants. Results show that our model generates more socially appropriate behavior compared to a state-of-theart model.

Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data

“…Later, various studies started to exploit the spatial configuration of the sensor/joint network. For instance, several data representations considered the relationships between sensors/joints [29,36,81], with network architectures designed to enable local processing of movement dynamics [13,19,39,40,44,75,80]. Performance improvements achieved by these methods suggest that body configuration information is important for activity recognition.…”

Automatic Assessment and Learning of Robot Social Abilities

“…Lathuilière et al [11] uses Deep RL to learn a gaze policy from an intrinsic reward function based on the audiovisual position of people with respect to the robot camera field of view. Gao et al [8] learns a robot policy for approaching groups of people by maximizing a group formation score and minimizing the displacement of other participants in the group when the robot approaches.…”

From Humans and Back: a Survey on Using Machine Learning to both Socially Perceive Humans and Explain to Them Robot Behaviours

“…Without having studies which understand how human beings interpret the behaviour of robotic systems and what are their expectations of such systems, it would be impossible, for example, to help humans trust their robot counterparts in critical and non-critical situations [15]. On this subject, Han et al [16] proposed a literature review, while interesting studies from a computer science, artificial intelligence, cognitive psychology, and philosophical [29] Automated rationale generation Recurrent neural networks [26] Goal communication, favourizing user anticipation Inverse reinforcement learning [63] Training personalized policies for teaching Model-free affective reinforcement learning Tega [67] Teaching a robot to learn toy names and the locations Reinforcement learning Kasper [42] Robot approaching behaviour to groups of humans Proximal policy optimization Pepper [35] Adaptively decide a monitoring distance and an approaching direction to improve user activity recognition performance perspective can be found in a recent full-day workshop [17]. For example, the Theory of Mind (ToM) model is used by both human and robot in order to understand each other in an autonomous car driving on the highway scenario [18].…”

“…Online learning of repeated stochastic games has also been suggested [40] as an efficient method to learn to collaborate with people through cheap talk [41]. In Gao et al [42], proximal policy optimization (PPO) was used to learn robot approaching behaviour, while in Vitiello et al [43], the robot proxemics is adapted with a neuro-fuzzy-Bayesian system.…”