Shut up and play: A musical approach to engagement and social presence in Human Robot Interaction

Abstract

With a view to studying the development of social relationships between humans and robots, it is our contention that music can help provide extended engagement and open ended interaction. In this paper we explore the effectiveness of music as a mode of engagement using Mortimer, a robot able to play a drum kit and employing a composition algorithm to respond to a human pianist. We used this system to conduct a comparative study into the effects of presenting the robot as a social actor or as an instrument. Using automated behavioural metrics, including face tracking, we found that participants in the social actor condition played for longer uninterrupted and stopped the robot mid-performance less. They also spent more time looking at the robot when not playing and less time looking at the piano when playing. We suggest these results indicate greater fluency of playing and engagement and more feelings of social presence towards the robot when presented as a social actor The 23rd IEEE International Symposium on Robot and Human Interactive Communication

Face the Music and Glance

“…Unlike our previous study, where all but one of the sessions were of equal length [23], the length of sessions ranged from the minimum of 20 minutes right up towards the maximum of 45. As such, the measure of tracks per session used previously is confounded by this variable and not a particularly elucidating one when attempting to investigate the smoothness and immersion of the participant's playing.…”

“…We examine the number of button stops, as opposed to allowing tracks to finish naturally or due to silence, as our earlier work had indicated that those in the reduced social condition used the tablet to explicitly stop the robot more than the those who experienced Mortimer presented as a social actor [23]. They also found that framing the study as a social interaction, and so dividing the session into social and musical interactions, increased both engagement and social presence.…”

“…A fuller explanation of the exactitudes of the algorithm and the motivation for the particular approach is available in [23]. A brief description follows.…”

“…Building on our previous findings that framing a session between pianists and their drumming robot, Mortimer, as a social interaction resulted in greater feelings of engagement and social presence [23], we present a study in which we investigate the inclusion of further social modalities, namely, head movements and facial expressions.…”

“…Following the methodology developed in our previous research, we rely mainly on automated behavioural metrics [23], analysing data logs to see how participants interacted with the robot and using face tracking to determine where they are focussing their attention during the sessions. In relation to a control group, we expected the use of head movements and facial expressions to increase social presence and engagement within the sessions, seen by increased session time, smoother playing and more displays of behaviour indicative of an interpersonal relationship.…”