Hierarchical Task-Parameterized Learning from Demonstration for Collaborative Object Movement

Abstract

Learning from demonstration (LfD) enables a robot to emulate natural human movement instead of merely executing preprogrammed behaviors. This article presents a hierarchical LfD structure of task-parameterized models for object movement tasks, which are ubiquitous in everyday life and could benefit from robotic support. Our approach uses the task-parameterized Gaussian mixture model (TP-GMM) algorithm to encode sets of demonstrations in separate models that each correspond to a different task situation. The robot then maximizes its expected performance in a new situation by either selecting a good existing model or requesting new demonstrations. Compared to a standard implementation that encodes all demonstrations together for all test situations, the proposed approach offers four advantages. First, a simply defined distance function can be used to estimate test performance by calculating the similarity between a test situation and the existing models. Second, the proposed approach can improve generalization, e.g., better satisfying the demonstrated task constraints and speeding up task execution. Third, because the hierarchical structure encodes each demonstrated situation individually, a wider range of task situations can be modeled in the same framework without deteriorating performance. Last, adding or removing demonstrations incurs low computational load, and thus, the robot's skill library can be built incrementally. We first instantiate the proposed approach in a simulated task to validate these advantages. We then show that the advantages transfer to real hardware for a task where naive participants collaborated with a Willow Garage PR2 robot to move a handheld object. For most tested scenarios, our hierarchical method achieved significantly better task performance and subjective ratings than both a passive model with only gravity compensation and a single TP-GMM encoding all demonstrations.

Robotics for Occupational Therapy: Learning Upper-Limb Exercises From Demonstrations

“…2) PP Exercise: For the time-based approach, we use the GMM/GMR approach from Section S-II, but with time as the input and the robot end-effector's position as the output. Note that to avoid placing too many data points where the object is moved slowly, we resample time and the trajectories based on trajectory length, as in our prior work [34]. Fig.…”

“…For the state-based model, we model the desired robot end-effector position based on its current position [34], as shown in Fig. 6.…”

“…As in our prior work [34], training data was collected where the expert (therapist) acted as the teacher and stood to the right of the robot while holding the robot's right arm, and the stroke survivor acted as the partner and used their impaired arm to hold the object with the robot's left arm. Force feedback between the two robot arms was achieved via a joint-level proportional-derivative (PD) controller, which enabled the therapist to adjust the robot motion while feeling how well the stroke survivor was doing the exercise.…”

“…The output dimensions are generally ahead of the input dimensions, so the robot will appear active and lead the movement. from sample demonstrations of the 1D and 2D exercises.For the state-based approach, we use Hu and Kuchenbecker's GMM/GMR approach[34] for each relevant dimension (y or z directions): at training time, the robot end-effector's position, velocity, and effort are encoded jointly by a GMM, and the effort is regressed from the robot's instantaneous position and velocity using GMR during autonomous execution. Details of using GMM/GMR are described in Section S-II in Supplemental Materials.…”

“…Fig.4shows examples of fitted Gaussian clusters; the position and velocity dimensions resemble a typical harmonic oscillator's phase plot, and the effort dimension depends on both position and velocity.2) PP Exercise: For the time-based approach, we use the GMM/GMR approach from Section S-II, but with time as the input and the robot end-effector's position as the output. Note that to avoid placing too many data points where the object is moved slowly, we resample time and the trajectories based on trajectory length, as in our prior work[34]. Fig.5shows results from sample demonstrations of the PP exercise.For the state-based model, we model the desired robot end-effector position based on its current position[34], as shown in Fig.6.…”