Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-toreal domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.

No abstract

Although motor primitives (MPs) have been studied extensively, much less attention has been devoted to studying their generalization to new situations. To cope with varying conditions, a MP's policy encoding must support generalization over task parameters to avoid learning separate primitives for each condition. Local and linear parameterized models have been proposed to interpolate over task parameters to provide limited generalization.In this paper, we present a global parametric motion primitive which allows generalization beyond local or linear models. Primitives are modelled using a linear basis function model with global non-linear basis functions. Using the global parametric model, we developed an online incremental learning framework for constructing a database of MPs from a single human demonstration. Above all, we propose a model selection method that can choose an optimal model complexity even with few training samples, which makes it suitable for online incremental learning. Experiments with a ball-in-a-cup task with varying string lengths demonstrate that the global parametric approach can successfully extract underlying regularities in a database of MPs leading to enhanced generalization capability of the parametric MPs and increased speed (convergence rate) of learning. Furthermore, it significantly excels over locally weighted regression both in terms of inter-and extrapolation.

No abstract

Although motor primitives (MPs) have been studied extensively, much less attention has been devoted to studying their generalization to new situations. To cope with varying conditions, a MP's policy encoding must support generalization over task parameters to avoid learning separate primitives for each condition. Local and linear parameterized models have been proposed to interpolate over task parameters to provide limited generalization. In this paper, we present a global parametric motion primitive (GPDMP) which allows generalization beyond local or linear models. Primitives are modeled using a linear basis function model with global non-linear basis functions. The model is constructed from initial non-parametric primitives found using a single human demonstration and subsequent episodes of reinforcement learning to adapt the demonstrated skill to other task parameters. The initial models are then used to optimize the parameters of the global parametric model. Experiments with a ball-in-a-cup task with varying string lengths show that GPDMP allows greatly improved extrapolation compared to earlier local or linear models.