Kinesthetic teaching and attentional supervision of structured tasks in human–robot interaction

Abstract

We present a framework that allows a robot manipulator to learn how to execute structured tasks from human demonstrations. The proposed system combines physical human-robot interaction with attentional supervision in order to support kinesthetic teaching, incremental learning, and cooperative execution of hierarchically structured tasks. In the proposed framework, the human demonstration is automatically segmented into basic movements, which are related to a task structure by an attentional system that supervises the overall interaction. The attentional system permits to track the human demonstration at different levels of abstraction and supports implicit non-verbal communication both during the teaching and the execution phase. Attention manipulation mechanisms (e.g. object and verbal cueing) can be exploited by the teacher to facilitate the learning process. On the other hand, the attentional system permits flexible and cooperative task execution. The paper describes the overall system architecture and details how cooperative tasks are learned and executed. The proposed approach is evaluated in a human-robot co-working scenario, showing that the robot is effectively able to rapidly learn and flexibly execute structured tasks.

Merging Position and orientation Motion Primitives

“…Complex robotic tasks, consisting of several actions, can be obtained by sequencing multiple motion primitives [22]- [26]. As in [24]- [26], this work represents the motion primitives as Dynamic Movement Primitives (DMP) [5], but other choices are possible [22], [23]. Given a set of DMPs, the problem arises of how the DMPs can be merged to generate a unique and smooth trajectory without stopping at the end of each motion primitive.…”

Intuitive Programming of Conditional Tasks by Demonstration of Multiple Solutions

“…In both works, preconditions and effects of symbolic actions are learned from demonstration but the corresponding symbols need to be manually defined. In [14], complex tasks are learned by kinesthetic teaching involving multiple visually tracked objects, where the task structure itself has to be predefined beforehand.…”

Symbolic Task Compression in Structured Task Learning

“…A drawback of the aforementioned approaches for task structure learning is that they rely on a set of pre-programmed primitive movements for robot execution. The framework presented in our previous works [11], [12], instead, enables simultaneous segmentation and labeling of the human demonstration exploiting supervisory attention mechanisms [13]- [16] to relate the labeled actions to a partially specified task structure. Concurrently, segmented trajectories are encoded into dynamical systems used to generate robot commands.…”

“…In this setting, the number of nodes/behaviors may easily grow with the number of activities affecting the system performance. For this purpose, this paper extends the framework by [11], [12] to reduce the number of nodes in a learned structure, while preserving a consistent execution. Our method explores the task structure searching for sequential behaviors to merge, while keeping task execution consistency.…”