Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots

Asfour, Tamim; Gyarfas, F.; Azad, Pedram; Dillmann, Rüdiger

doi:10.1109/ichr.2006.321361

Cited by 97 publications

(41 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, a high number of training sequences is needed. The same holds for approaches which mainly focus on the reproduction of whole body, respectively, arm movements using learned HMMs (e.g., [14], [15], [16]). …”

Section: Related Workmentioning

confidence: 99%

Recognizing complex, parameterized gestures from monocular image sequences

Axenbeck

Bennewitz

Behnke

et al. 2008

Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots

View full text Add to dashboard Cite

Abstract-Robotic assistants designed to coexist and communicate with humans in the real world should be able to interact with them in an intuitive way. This requires that the robots are able to recognize typical gestures performed by humans such as head shaking/nodding, hand waving, or pointing. In this paper, we present a system that is able to spot and recognize complex, parameterized gestures from monocular image sequences. To represent people, we locate their faces and hands using trained classifiers and track them over time. We use few, expressive features extracted out of this compact representation as input to hidden Markov models (HMMs). First, we segment gestures into distinct phases and train HMMs for each phase separately. Then, we construct composed HMMs, which consist of the individual phase-HMMs. Once a specific phase is recognized, we estimate the parameter of the current gesture, e.g., the target of a pointing gesture. As we demonstrate in the experiments, our method is able to robustly locate and track hands, despite of the fact that they can take a large number of substantially different shapes. Based on this, our system is able to reliably spot and recognize a variety of complex, parameterized gestures.

show abstract

Section: Related Workmentioning

confidence: 99%

Recognizing complex, parameterized gestures from monocular image sequences

Axenbeck

Bennewitz

Behnke

et al. 2008

Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots

View full text Add to dashboard Cite

show abstract

“…Other authors proposed to use hidden Markov models (HMM) for encoding the temporal sequence of manipulation goals (Asfour et al, 2006;Calinon et al, 2005;Tso and Liu, 1996). Pardowitz and Dillmann (2007) presented a system that generalizes over household tasks in a hierarchical manner.…”

Section: Related Workmentioning

confidence: 99%

Approaches to Probabilistic Model Learning for Mobile Manipulation Robots

Sturm¹

2013

Springer Tracts in Advanced Robotics

View full text Add to dashboard Cite

Mobile manipulation robots are envisioned to provide many useful services both in domestic environments as well as in the industrial context. Examples include domestic service robots, that implement large parts of the housework, and versatile industrial assistants, that provide automation, transportation, inspection, and monitoring services. The challenge in these applications is that the robots have to function under changing, real-world conditions, be able to deal with considerable amounts of noise and uncertainty, and operate without the supervision of an expert. To meet these challenges, current robotic systems are typically custom-tailored to specific applications in welldefined environments, and therefore cannot deal robustly with changes in the situation. This thesis presents novel learning techniques that enable mobile manipulation robots, i.e., mobile platforms with one or more robotic manipulators, to autonomously adapt to new or changing situations. The developed approaches in this thesis cover the following four topics: (1) learning the robot's kinematic structure and properties using actuation and visual feedback, (2) learning about articulated objects in the environment in which the robot is operating, (3) using tactile feedback to augment the visual perception, and (4) learning novel manipulation tasks from human demonstrations.In the first part of this thesis, we present innovative approaches to learning a robot's own body schema from scratch using visual self-observation. This allows manipulation robots to calibrate themselves automatically and to adapt their body schemata autonomously, for example after hardware failures or during tool use. In the second part, we extend the developed framework to learning about articulated objects -such as doors and drawers -with which service robots often need to interact. The presented algorithms enable robots to learn accurate kinematic models of articulated objects, which in turn allow them to interact with the objects robustly. In the third part, we provide approaches that allow manipulation robots to make use of tactile perception -an ability that is known to play an important role in human object manipulation skills. The main contributions in this part are approaches to identifying objects and to perceiving aspects of their internal states. With this, a manipulation robot can verify that it has grasped the correct object and, for example, discriminate full from empty bottles. Finally, we present an integrated system that allows human operators to intuitively teach a robot novel manipulation tasks by demonstration.All techniques developed in the thesis are based on probabilistic learning and inference. They have been implemented and evaluated on real robots as well as in simulation. Extensive experiments have been conducted to analyze and validate the properties of the developed algorithms and to demonstrate a significant increase in robustness, adaptability, and utility of mobile manipulation robots in everyday life. Zusammenfassung

show abstract

“…Within this project Dillmann et al [8] evaluated different methods for interactive natural programming of robots and implemented a programming by demonstration system for dual-arm manipulation tasks using petri nets for task coordination [14]. The work on imitation learning of dual-arm manipulation continues in the Collaborative Research Center (SFB) 588 on humanoid robots [15].…”

Section: Related Workmentioning

confidence: 99%

Spatial Programming for Industrial Robots through Task Demonstration

Lambrecht

Kleinsorge

Rosenstrauch

et al. 2013

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

We present an intuitive system for the programming of industrial robots using markerless gesture recognition and mobile augmented reality in terms of programming by demonstration. The approach covers gesture-based task definition and adaption by human demonstration, as well as task evaluation through augmented reality. A 3D motion tracking system and a handheld device establish the basis for the presented spatial programming system. In this publication, we present a prototype toward the programming of an assembly sequence consisting of several pick-and-place tasks. A scene reconstruction provides pose estimation of known objects with the help of the 2D camera of the handheld. Therefore, the programmer is able to define the program through natural bare-hand manipulation of these objects with the help of direct visual feedback in the augmented reality application. The program can be adapted by gestures and transmitted subsequently to an arbitrary industrial robot controller using a unified interface. Finally, we discuss an application of the presented spatial programming approach toward robot-based welding tasks.

show abstract

Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots

Cited by 97 publications

References 14 publications

Recognizing complex, parameterized gestures from monocular image sequences

Recognizing complex, parameterized gestures from monocular image sequences

Approaches to Probabilistic Model Learning for Mobile Manipulation Robots

Spatial Programming for Industrial Robots through Task Demonstration

Contact Info

Product

Resources

About