Learning and generalization of complex tasks from unstructured demonstrations

Abstract

We present a novel method for segmenting demonstrations, recognizing repeated skills, and generalizing complex tasks from unstructured demonstrations. This method combines many of the advantages of recent automatic segmentation methods for learning from demonstration into a single principled, integrated framework. Specifically, we use the Beta Process Autoregressive Hidden Markov Model and Dynamic Movement Primitives to learn and generalize a multi-step task on the PR2 mobile manipulator and to demonstrate the potential of our framework to learn a large library of skills over time.

Probabilistic inference for determining options in reinforcement learning

“…Instead of learning policies over state-torque mappings to control robots, the agent learns parameters of a trajectory generator (Kober and Peters 2010;Kajita et al 2003). Based on a Beta-Process Autoregressive HMM proposed by Fox et al (2009), Niekum et al (2012 proposed a method to segment demonstrated trajectories into a sequence of primitives, addressing the imitation learning problem. Rosman and Konidaris (2015) extend the work of Fox et al (2009) to allow skill discovery in the inverse reinforcement learning setting.…”

Personalization Framework for Adaptive Robotic Feeding Assistance

“…The idea is similar to the User Tailoring one, though they apply it to improve the policy rather than to adapt a well-learned task to a specific user. A framework to learn and generalize complex tasks from unstructured demonstrations is proposed in Niekum et al [14]. The method is able to recognize repeated instances of skills and generalize them to new settings.…”

Learning motion primitives from demonstration

“…Compared to other programming methods, LfD has the advantages that it does not require expert knowledge of the domain dynamics so as to avoid the problem of performance brittleness resulting from model simplifications, and non-robotics experts can participate in the development of control strategy for LfD does not require any relevant expert domain knowledge. [8][9][10] However, demonstrations are often simply treated as trajectories to be mimicked to accomplish the specific tasks, 11,12 losing the information of the task or the related environment. So, it is difficult for the learned policies to generalize well to the new situations, especially for the complex tasks.…”

“…To obtain the effective and reasonable segments requires prior knowledge of the robot's internal representations, kinematic properties, existing skill competencies, as well as task constraints. 11,12 The complex tasks in the unstructured working environment of WMRA which contain many repeated motion primitives also enhance the difficulty of manual segmentation. For the above reasons, much effort has been focused on automating the segmentation process.…”

“…Recently, S Niekum et al 11 adopted the beta process autoregressive HMM (BP-AR-HMM) algorithm proposed by E Fox et al, 19,20 which is a typical unsupervised learning approach, to segment a learned motion into a sequence of skills and mainly validated this approach in simulation environment. Additionally, the autoregressive HMMs (AR-HMM) can capture the temporal dependencies present in the data set.…”