Active Robot Learning for Temporal Task Models

Abstract

With the goal of having robots learn new skills after deployment, we propose an active learning framework for modelling user preferences about task execution. The proposed approach interactively gathers information by asking questions expressed in natural language. We study the validity and the learning performance of the proposed approach and two of its variants compared to a passive learning strategy. We further investigate the human-robotinteraction nature of the framework conducting a usability study with 18 subjects. The results show that active strategies are applicable for learning preferences in temporal tasks from non-expert users. Furthermore, the results provide insights in the interaction design of active learning robots. CCS CONCEPTS • Computing methodologies → Active learning settings; • Human-centered computing → Natural language interfaces; Interaction design; • Computer systems organization → External interfaces for robotics;

Teacher-Aware Active Robot Learning

“…This wide range of preferences suggests how the best strategy is likely to be user-dependent, based e.g. on their patience or their teaching skills as observed also in [9], [12], [28], [43].…”

“…Hayes and Scassellati [20] enabled a robot to ask questions during the execution of a task, in order to obtain information about the feasibility of the next step. Similarly, Racca and Kyrki [12] proposed an AL technique for learning task models by combining LfD and queries expressed in natural language. Sadigh et al [21] presented an application of AL in an Inverse Reinforcement Learning (IRL) scenario, where an autonomous car learned from users their preferred driving style by posing comparison queries.…”

“…Another line of research looks instead at the nature of the interaction with AL robots. Aspects like the nature of robot's queries [10] and of the query selection mechanism [12], [24], and their impact on the quality of user's answers [25], [26] have been studied. Also the balance of control between learner and teacher during the interaction [9], the effect of queries' timing [27] and the transparency of the learning process [28] have been investigated.…”

“…Algorithm 1 summarizes the tree building process, starting from the Entity set E and a desired root of the tree. On the Entity-Category tree T , we compute the relevance scores w c,e as w c,e = exp(−γd(c, e)), Add ρ to T 10: end for 11: Prune trivial nodes from T (nodes with a single child) 12: C ← all non-terminal nodes of T where d(c, e) is a distance metric defined as the number of edges between nodes e and c in T and γ is a scale parameter.…”

Interactive Tuning of Robot Program Parameters via Expected Divergence Maximization

“…This is particularly important for problems where collecting labels is costly, as in robotics and HRI where AL has seen growing interest. AL techniques have been used to economically learn robot policies [21,22,55,62] and task representations [25,34,41,53], to guide efficient information gathering [17], and to learn reward functions by querying the user with different type of queries [9,10,24,59]. Another line of research has instead concentrated on the interactive nature of AL, with work investigating the design of active robot learners [18,19], the ability of users to answer the robot's questions [32,57] and the development of AL strategies that take into account the user in their query selection [11,38,54].…”