Endowing Robots with Longer-term Autonomy by Recovering from External Disturbances in Manipulation through Grounded Anomaly Classification and Recovery Policies

Abstract

Robot manipulation is increasingly poised to interact with humans in co-shared workspaces. Despite increasingly robust manipulation and control algorithms, failure modes continue to exist whenever models do not capture the dynamics of the unstructured environment. To obtain longer-term horizons in robot automation, robots must develop introspection and recovery abilities. We contribute a set of recovery policies to deal with anomalies produced by external disturbances as well as anomaly classification through the use of non-parametric statistics with memoized variational inference with scalable adaptation. A recovery critic stands atop of a tightly-integrated, graph-based online motion-generation and introspection system that resolves a wide range of anomalous situations. Policies, skills, and introspection models are learned incrementally and contextually in a task. Two task-level recovery policies: re-enactment and adaptation resolve accidental and persistent anomalies respectively. Re-enactment policies model human decision making to reenact the best skill in the task-graph. Adaptive recoveries leverage human intuition about the task-state to overcome persistent errors. The system is capable of fast and robust anomaly identification and classification during all phases of a task including during the execution of newly learned recovery skills. The introspection system uses non-parametric priors along with Markov jump linear systems and memoized variational inference with scalable adaptation to learn a model from the data in an incremental way and yield compact interpretable models that enhance classification and identification accuracy. Extensive real-robot experimentation with various strenuous anomalous conditions in a co-bot scenario is induced and resolved at different phases of a task and in different combinations. The system executes around-the-clock introspection and recovery and even elicited self-recovery when misclassifications occurred.

Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery

“…Plan repair for failures that occur during execution require a fault diagnosis [23] and there is a large body of ongoing work in robotics focused on fault diagnosis techniques [29]. These works use a range of methods, including unsatisfied preconditions [10,15,31], first-order logic inference [50], case-based reasoning [23,36], sensor signal processing [1,17,28], Bayes nets and Dynamic Bayesian Networks [8,30], Hidden Markov Models (HMMs) [47], particle filters [44,53], and neural networks [35,37] to diagnose failures. Depending on the context of the work, the diagnosis either identifies what is wrong with the robot-e.g., object not visible [31] or sonar is blind [17]-or why it is wrong-e.g., there was a collision with the environment [35].…”

Non-Markov Policies to Reduce Sequential Failures in Robot Bin Picking

“…Extensive work has also been done in the scope of error recovery. Wu et al [32] explore a policy by which the robot learns via demonstration after making an error. Wang et al [29] use a multimodal transition model learned through reinforcement learning to improve the success rate of robots in unstructured environments.…”