A modular approach to learning manipulation strategies from human demonstration

Abstract

Object manipulation is a challenging task for robotics, as the physics involved in object interaction is complex and hard to express analytically. Here we introduce a modular approach for learning a manipulation strategy from human demonstration. Firstly we record a human performing a task that requires an adaptive control strategy in different conditions, i.e. different task contexts. We then perform modular decomposition of the control strategy, using phases of the recorded actions to guide segmentation. Each module represents a part of the strategy, encoded as a pair of forward and inverse models. All modules contribute to the final control policy; their recommendations are integrated via a system of weighting based on their own estimated error in the current task context. We validate our approach by demonstrating it, both in a simulation for clarity, and on a real robot platform to demonstrate robustness and capacity to generalise. The robot task is opening bottle caps. We show that our approach can modularize an adaptive control strat-

Learning Sequential Force Interaction Skills

“…By modifying and recombining existing MPs, they are able to represent complex trajectories. Other approaches try to identify MPs using Bayesian Binning [6], Transition State Clustering [7], Conditional Random Fields [8,9], Gaussian Mixture Models [10][11][12], Hidden Markov Models [13,14] or Temporal Alignment for Control [15]. Changepoint methods can be used for decomposing a task in a two-step process.…”

“…Conflicts of Interest: The authors declare no conflict of interest.Abbreviations The following abbreviations are used in this manuscript: DMP Dynamic Movement Primitive DND Directional Normal Distribution DoF Degree of freedom DS Dynamical System EM Expectation-Maximization where the entries of the matrix R VS (t) are R (0,0)VS (t) = (1 − a 2 x ) cos(t) + a 2 x , R (0,1) VS (t) = −a x a y cos(t) + a z sin(t) + a x a y , R (0,2) VS (t) = −a x a z cos(t) − a y sin(t) + a x a z , R (1,0) VS (t) = −a x a y cos(t) − a z sin(t) + a x a y , R (1,1) VS (t) = (1 − a 2 y ) cos(t) + a2 y , VS (t) = −a y a z cos(t) + a x sin(t) + a y a z , R (2,0) VS (t) = −a x a z cos(t) + a y sin(t) + a x a z , R (2,1) VS (t) = −a y a z cos(t) − a x sin(t) + a y a z , R (2,2)VS (t) = (1 − a 2 z ) cos(t) + a 2 z , with R (i,j)VS (t) being the entry of the ith row and jth column. The matrix can be written in the formR VS (t) = D cos(t) + E sin(t) + F with D =    1 − a 2 x −a x a y −a x a z −a x a y 1 − a 2 y −a y a z −a x a z −a y a z 1 − a 2 VI (t) = R VS (t)R SI = (D cos(t) + E sin(t) + F) = DR SI cos(t) + ER SI sin(t) + FR SI .The values of the constant vectors d, e and f used in(11) can then be found by multiplying the matrices and vectorizing the resulting matrices.R(0,0) FV = R FS (0, 0)(a 2…”

Data-Driven Model Predictive Control for the Contact-Rich Task of Food Cutting

“…In an attempt to overcome the difficulties of classical control, several categories of data-driven approaches have emerged in manipulation. Some of them, employ human demonstrations in order to learn the force profiles needed to successfully complete complex tasks [12]- [17]. However, in our scenario, recording a demonstration by leading the robot through the motion, would prove problematic since it would be impossible to distinguish the wrench applied by the human from the one applied by the object without having to resort in external tracking solutions as the one in [18].…”

Learning Force‐Relevant Skills from Human Demonstration

“…the force applied on the object was measured with a high resolution tactile sensor. A data glove mounted with tactile sensing was used for direct demonstration in [24]. Based on the similarity of varying demonstrations, the learned forcebased skills were modularized and can be further combined for more complicated tasks.…”