Learning from Physical Human Corrections, One Feature at a Time

Abstract

We focus on learning robot objective functions from human guidance: specifically, from physical corrections provided by the person while the robot is acting. Objective functions are typically parametrized in terms of features, which capture aspects of the task that might be important. When the person intervenes to correct the robot's behavior, the robot should update its understanding of which features matter, how much, and in what way. Unfortunately, real users do not provide optimal corrections that isolate exactly what the robot was doing wrong. Thus, when receiving a correction, it is difficult for the robot to determine which features the person meant to correct, and which features were changed unintentionally. In this paper, we propose to improve the efficiency of robot learning during physical interactions by reducing unintended learning. Our approach allows the human-robot team to focus on learning one feature at a time, unlike state-of-the-art techniques that update all features at once. We derive an online method for identifying the single feature which the human is trying to change during physical interaction, and experimentally compare this one-at-a-time approach to the all-at-once baseline in a user study. Our results suggest that users teaching one-at-a-time perform better, especially in tasks that require changing multiple features.

Learning the Correct Robot Trajectory in Real-Time from Physical Human Interactions

“…Replanning optimal trajectories. If we treat physical human interactions as observations about an underlying objective function, then the robot updates its objective after each interaction and replans an optimal trajectory with respect to this new objective [4,5,19]. Jain et al [19] present one such formulation, where the human physically corrects a robot waypoint offline, and the robot solves for its optimal trajectory at the next trial.…”

“…Jain et al [19] present one such formulation, where the human physically corrects a robot waypoint offline, and the robot solves for its optimal trajectory at the next trial. Although we have extended this method for real-time implementation [4,5], the approach is practically limited to only a few features and simple tasks, since optimal trajectory replanning for more complex settings cannot be performed in less than a second [21,39]. Accordingly, we treat trajectory replanning as an optimal offline solution and assess how much performance the robot loses using our real-time (i.e., sub-millisecond) approach.…”

Learning and Composing Primitive Skills for Dual-Arm Manipulation

“…With this in mind, this work avoids demonstrating a task itself but, instead, teaches the robot the involved primitive skills. This task factorisation provides similar benefits as the work in (Bajcsy et al 2018): it allows the user to teach one feature of the task at a time, and, if required, to correct them individually.…”

“…the issue of identifying a mapping between the teacher and the learner which allows transferring of information from one to the other (Dautenhahn and Nehaniv 2002). Moreover, complex motions involve a mixture of human intentions, which are difficult to accurately learn when following an all-at-once learning baseline (Bajcsy et al 2018). On top of that, teaching a dual-arm system can suppose a high endeavour for non-robotics-experts (Akgun et al 2012).…”

Characterizing Input Methods for Human-to-Robot Demonstrations

“…5) Incremental Learning from Demonstration (ILfD): ILfD refers to a special case where the demonstrations are provided during the learning process. ILfD has similar needs to LfD, but may require demonstrations that are responsive to problems identified by the user (e.g., [20]) or partial task demonstrations.…”