Generalizing topological task graphs from multiple symbolic demonstrations in programming by demonstration (PbD) processes

Abstract

Many programming by demonstration methods encode demonstrations into sequences of predefined symbols and then build a generalized task structure such as a topological graph. The longest common subsequence (LCS) algorithm is one of the potential techniques to help build generalized task structures from multiple sequences. However the LCS problem is NP hard, so a couple of suboptimal LCS approaches have been adopted in the past, involving a pair-wise comparison of sequences or a search for the common symbols within a small window. This paper argues that an LCS of multiple sequences results in a better generalization than pairwise comparison, and in many practical situations it is feasible to find an LCS of multiple sequences. So a novel LCS finding algorithm is presented for applications in the programming by demonstration domain. The algorithm has been extensively tested for sequences of random symbols and its application in a path planning example is presented. I. BACKGROUNDService robots are one of the ultimate goals of robotics. Having successful applications in industries, robotics is more focussed to bring intelligent robotic assistance in our daily life. We all have changing and different activities at changing and different work places, and a robot requires different programs for the different activities and work places. It is infeasible for robot developers to foresee all the possible scenarios and preprogram a robot. End-user robot programming is a potential solution to this robot programming problem [1]- [4] i.e. it would be interesting if a robot could be instructed by its users. However robot users are not expected to have sufficient skills to program the robot. Recently, programming by demonstration (PbD) has demonstrated significant potential and gained considerable recognition amongst end-user robot programming approaches.In a PbD process, a robot is instructed through demonstrations of example tasks. The robot records observations through various sensors while the demonstrator performs the example tasks. The recorded demonstrations are then processed to build a generalized task model. Various factors, such as variations in the workspace and flexibility in the selection of actions, induce variability in different demonstrations of the same task. The task generalization process exploits the variability in the demonstrations for inferencing and hence better learning than pure imitation [2], [4]- [6].For efficient learning of task plans consisting of several moves and interactions, many PbD processes encode the demonstrations into sequences of predefined symbols and then a generalized task structure, such as a hierarchy [6] or a topological graph [5], is built from multiple demonstrations. The symbols represent elementary moves and/or interactions (called primitives). The symbols could have parameters such as goto(x,y) and move-forward(1m) [5], [7], resulting in an infinite set of symbols' alphabet for a PbD system. The process of building a task structure representing all the demonstrated sequences is...

Skills for vision-based applications in robotics application to aeronautics assembly pilot station

“…Skill-based programming is one approach to alleviate this drawback. The idea is modelling system capabilities in simple and intuitive symbolic units [6] [7]. Explaining this paradigm is easier with a practical example: in the industry lot sizes are smaller and smaller, and as a consequence, costs of reprogramming the robots grows.…”

Intuitive Programming of Conditional Tasks by Demonstration of Multiple Solutions

“…Instead, this is not needed with the presented approach. In [11], multiple demonstrations are encoded in sequences of predefined symbols to find the longest common subsequence. Symbolic actions for planning are extracted from demonstrations in [12] and [13].…”

Symbolic Task Compression in Structured Task Learning

“…In order to simplify robot programming, researchers focused on learning tasks from human demonstrations and contextual information [1]- [9]. These approaches are effective in learning symbolic and robot-independent task representations from observation.…”