Learning complex sequential tasks from demonstration: A pizza dough rolling case study

Abstract

This paper introduces a hierarchical framework that is capable of learning complex sequential tasks from human demonstrations through kinesthetic teaching, with minimal human intervention. Via an automatic task segmentation and action primitive discovery algorithm, we are able to learn both the high-level task decomposition (into action primitives), as well as low-level motion parameterizations for each action, in a fully integrated framework. In order to reach the desired task goal, we encode a task metric based on the evolution of the manipulated object during demonstration, and use it to sequence and parametrize each action primitive. We illustrate this framework with a pizza dough rolling task and show how the learned hierarchical knowledge is directly used for autonomous robot execution.

A Dynamical-System-Based Approach for Controlling Robotic Manipulators During Noncontact/Contact Transitions

“…. , d} and the rate of change of (17) and (19) are the same, the motion reaches x c when t = +∞, the dynamical system (1) and (2) with respect to (5) and (6) contacts the surface at x c ; i.e. Objective 2 is satisfied.…”

“…lim t→+∞ q 1 Tẋ = lim t→+∞ e −tω (q 1 Tẋ 0 − (q 1 T x 0 ω + q 1 Tẋ 0 )ωt) = 0. Hence, the motion generated by 1 and 2 with respect to (4) and (5), enters the contact surface with zero normal velocity. Hence, Objective 1 is satisfied.…”

“…Establishing a stable contact with an environment is the first step toward accomplishing interactive tasks. A wide variety of many real-world manipulation tasks, such as milling/polishing/finishing workpieces [1], [2], wiping/painting surfaces [3], [4], peeling or dough rolling [5], include interactions between a tool and an environment. For such applications, the complete scenario can be categorized into three regions: (I) Moving in free motion space and approaching the contact surface; i.e.…”

Robot Choreography: The Use of the Kinetography Laban System to Notate Robot Action and Motion

“…By using a nonlinear differential equation to be learned to represent an observed movement, the researchers built a library of movements by labeling each recorded movement according to task and con text (e.g., grasping, placing, and realizing). In [13], a hierarchical framework capable of learning complex sequential tasks from human demonstrations was proposed. Through a task-segmentation and action-primitive discovery algorithm, both the high-level task decomposition and low-level motion parameterizations were achieved for each action.…”

Learning Augmented Joint-Space Task-Oriented Dynamical Systems: A Linear Parameter Varying and Synergetic Control Approach

“…Task-space motion generators learned from demonstrations all rely on projecting the desired task-space velocity into joint-space via Jacobian Pseudo-Inverse IK approximations and variants thereof [1]. When the main focus is on executing a specific task-space behavior, regardless of a joint-space constraint, this approach is sufficient [13], [14]. However, for other applications, such an approach yields significant problems [15].…”