End-User Programming of Low-and High-Level Actions for Robotic Task Planning

Abstract

Programming robots for general purpose applications is extremely challenging due to the great diversity of end-user tasks ranging from manufacturing environments to personal homes. Recent work has focused on enabling endusers to program robots using Programming by Demonstration. However, teaching robots new actions from scratch that can be reused for unseen tasks remains a difficult challenge and is generally left up to robotic experts. We propose iRoPro, an interactive Robot Programming framework that allows endusers to teach robots new actions from scratch and reuse them with a task planner. In this work we provide a system implementation on a two-armed Baxter robot that (i) allows simultaneous teaching of low-and high-level actions by demonstration, (ii) includes a user interface for action creation with condition inference and modification, and (iii) allows creating and solving previously unseen problems using a task planner for the robot to execute in real-time. We evaluate the generalisation power of the system on six benchmark tasks and show how taught actions can be easily reused for complex tasks. We further demonstrate its usability with a user study (N=21), where users completed eight tasks to teach the robot new actions that are reused with a task planner. The study demonstrates that users with any programming level and educational background can easily learn and use the system.

iRoPro: An interactive Robot Programming Framework

“…3 Experimental setup of the user study. Users programmed the Baxter robot via a graphical interface in order to manipulate objects (shown with their type hierarchies) in the task domain [28].…”

“…In the following sections we give a brief overview of the experimental setup, design, measurements and results. Further details on all experiments can be found in previous work [26,28].…”

“…5 In the following sections we will give a brief overview of the system implementation. Further details can be found in our previous work [28].…”

“…We captured the following data during the experiments: Table 1 Benchmark tasks for the user study where the first two tasks were used to introduce participants to the system [28]. The pre-study test included seven questions related to their understanding of the concepts presented at the start of the experiment, e.g., syntax ('If move(CUBE) describes a move action, tick all statements that are true.…”

A Survey on End-User Robot Programming

“…Many systems focus on enabling users to specify the structure and logic of a program using pre-specified robot primitives, such as grasps or spinning, that are often developed by expert robot programmers (e.g., [3,7,8,10,17,18,44,47,53,55,58,63,67,77,86,96,101,103,112,116,117]). Some end-user robot programs give the user more granular access by allowing end-users to specify the primitives used for subsequent programming (e.g., [34,55,68,69,108,112]) or by taking an end-to-end approach in which end-users are fully responsible for specifying every aspect of the program, as in imitation learning (e.g., [57,74]). On the other hand, other end-user robot programming systems do the bulk of the program specification, using techniques like automated planning (e.g., [68,69]).…”