Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees

Abstract

In dialogue, speakers continuously adapt their speech to accommodate the listener, based on the feedback they receive. In this paper, we explore the modelling of such behaviours in the context of a robot presenting a painting. A Behaviour Tree is used to organise the behaviour on different levels, and allow the robot to adapt its behaviour in real-time; the tree organises engagement, joint attention, turn-taking, feedback and incremental speech processing. An initial implementation of the model is presented, and the system is evaluated in a user study, where the adaptive robot presenter is compared to a non-adaptive version. The adaptive version is found to be more engaging by the users, although no effects are found on the retention of the presented material.

Multimodal User Feedback During Adaptive Robot-Human Presentations

“…For our analysis, we set up an experiment where participants interacted with a robot presenting a piece of art, as seen in Figure 1, similar to that used in Axelsson and Skantze (2019). As a robot platform, the Furhat robot head was used, which has a back-projected animated face and a mechanical neck (Al Moubayed et al, 2012).…”

“…In such settings, the presenter can be expected to have the turn the majority of the time, while the listener (the audience) provides positive and negative feedback to the presenter. In our previous work, we have shown how the agent can use such feedback to adapt the presentation in real time, using behaviour trees and knowledge graphs to model the grounding status of the information presented (Axelsson and Skantze, 2020), and that an agent that adapts its presentation according to the feedback it receives is preferred by users (Axelsson and Skantze, 2019). However, since we have so far only evaluated the system using either a Wizard of Oz setup (Axelsson and Skantze, 2019) or with simulated users (Axelsson and Skantze, 2020), we have not yet addressed the question of how feedback in this setting could be automatically identified and classified.…”

Modeling Feedback in Interaction With Conversational Agents—A Review

“…For systems that only handle one domain or type of task, like the calendar agent used as an example by Buschmeier, this minimal mentalizing approach might already be sufficient. For systems that present more arbitrary information, like the poster presenting system by Axelsson and Skantze (2019), negative understanding toward one part of the utterance may imply positive understanding toward another part, or perhaps invalidate the system's previous belief that the user understood something earlier, and thus feedback must also be interpreted and handled in terms of some kind of dialogue state or more complex partner model, which becomes similar to a representation of full common ground-with higher associated costs for language production and adaptation (Keysar, 1997).…”

Using knowledge graphs and behaviour trees for feedback-aware presentation agents

“…For each user in the interaction, individual knowledge graph properties are marked as either grounded or un-grounded, depending on the feedback that is received. The behaviour tree presented here is an extension of the one presented in a previous paper [5]. The extensions to the system consist of both changes to the tree and to the external components used to synchronise the tree's operation.…”

“…In a previous paper [5], we have shown how behaviour trees can be used to model the grounding process and the interaction between a robot presenting a piece of art to its audience. We used the art presentation scenario in an experiment where users showed a preference towards a mode that adapted to their feedback over one that used a pre-determined script [5]. One limitation of our previous model was the coarseness of its grounding model, and how it could only present and ground one pre-written utterance at a time.…”