Gesture-Based Extraction of Robot Skill Parameters for Intuitive Robot Programming

Abstract

Despite a lot of research in the field, only very little experience exists with Teaching by Demonstration (TbD) in actual industrial use cases. In the factory of the future, it is necessary to rapidly reprogram flexible mobile manipulators to perform new tasks, when the need arises, for which a working system capable of TbD would be ideal. Contrary to current TbD approaches, that generally aim to recognize both action and where it is applied, we propose a division of labor, where the operator manually specifies the action the robot should perform, while gestures are used for specifying the relevant action parameter (e.g. on which object to apply the action). Using this two-step method has the advantages that there is no uncertainty of which action the robot will perform, it takes into account that the environment changes, so objects do not need to be at predefined locations, and the parameter specification is possible even for inexperienced users. Experiments with 24 people in 3 different environments verify that it is indeed intuitive, even for a robotics novice, to program a mobile manipulator using this method. M. R. Pedersen ( ) · V. Krüger Robotics, Vision and Machine Intelligence (RVMI) Lab,

Use of human gestures for controlling a mobile robot via adaptive CMAC network and fuzzy logic controller

“…First, many existing approaches [15,17,18,19] prefer to apply qualitative commands to control mobile robots. These qualitative commands directly inform robots to move forwards or backwards, and turn left or right.…”

“…A number of approaches have been proposed to control robots using human gestures. For instance, Pedersen and Krüger defined five gestures used as control commands to guide the movements of mobile robots [15]; and a similar piece of work was also reported by Pentiuc et al [16]. Wongphati [17,18] used a set of body gestures to build a stationary of control commands for mobile robot control.…”

Automated disassembly of e-waste—requirements on modeling of processes and product states

“…Human behavior in solving these tasks often serves as a model for the design of skill systems that focus on this aspect. This in turn is also related to advanced forms of programming by demonstration (Pedersen and Krüger, 2015)-Another question is, what set of skills is necessary or useful to be used in a domain such as manufacturing? The result was concrete skills like "pick" and "place" (Pedersen et al, 2016).…”

Towards Tool-Support for Robot-Assisted Product Creation in Fab Labs

“…Pedersen and Krüger [11] propose an approach that links body poses to specific pre-programmed skills. These skills are higher level robot tasks that combine different robot actions and observations such as object detection.…”