Intuitive Programming of Conditional Tasks by Demonstration of Multiple Solutions

Abstract

Conditional tasks include a decision on how the robot should react to an observation. This requires to select the appropriate action during execution. For instance, spatial sorting of objects may require different goal positions based on the objects properties, such as weight or geometry. We propose a framework that allows a user to demonstrate conditional tasks including recovery behaviors for expected situations. In our framework, human demonstrations define the required actions for task completion, which we term solutions. Each specific solution accounts for different conditions which may arise during execution. We exploit a clustering scheme to assign multiple demonstrations to a specific solution, which is then encoded in a probabilistic model. At runtime, our approach monitors the execution of the current solution using measured robot pose, external wrench, and grasp status. Deviations from the expected state are then classified as anomalies. This triggers the execution of an alternative solution, appropriately selected from the pool of demonstrated actions. Experiments on a real robot show the capability of the proposed approach to detect anomalies online and switch to an appropriate solution that fulfills the task.

Collaborative Programming of Conditional Robot Tasks

“…Common methods for encoding a behavior require multiple user demonstrations of the same skill under slightly different environment conditions [6] [8] [10]. However, with the proposed framework we reduce the number of initial skill demonstrations.…”

“…Trajectories encoding takes place on a low level of the motion and tries to directly retrieve a non-linear mapping between sensory observation and motor action. For this, a large body of work uses skill encoding based on statistical modeling [10] [14] [15], dynamical systems [16] [17], or a combination of both [18] [19]. The algorithm developed in [18] extracts task constraints from a probabilistic model, used to modulate a dynamical system.…”

“…Kulić et al [15] use Hidden Markov Models to reproduce and recognize repeated motion primitives. The frameworks presented in [10] and [14] use time parameterized Gaussian Mixture Models (GMM) to probabilistically encode the demonstrations and Gaussian Mixture Regression (GMR) to condition the multivariate normal distributions on a time vector. Eiband et al [10] present a method, that allows active transitioning between skills of a solution pool when anomalies are detected during task execution.…”

“…While previously described methods [5] - [8] rely on the user to identify unintended or new situations, the approaches in [3], [4], [10] and [16] aim to automatically detect anomalies or task deficiencies. Maeda et al [3] propose a method based on Gaussian Processes to quantify the generalization capability of an extrapolated trajectory for an unseen situation.…”

“…A z-test, determines if the current execution is correlated with the successful examples. In contrast to that, in [10], the sequence of mean vectors and covariance matrices of the probabilistic skill encoding is used in every time step to compute a Mahalanobis distance, quantifying the deviation between measured and commanded state. Each of the above mentioned frameworks includes either a manual action-labeling or error threshold tuning.…”