Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery

Abstract

With the growing capabilities of intelligent systems, the integration of robots in our everyday life is increasing. However, when interacting in such complex human environments, the occasional failure of robotic systems is inevitable. The field of explainable AI has sought to make complex-decision making systems more interpretable but most existing techniques target domain experts. On the contrary, in many failure cases, robots will require recovery assistance from non-expert users. In this work, we introduce a new type of explanation, E 𝑒𝑟𝑟 , that explains the cause of an unexpected failure during an agent's plan execution to non-experts. In order for E 𝑒𝑟𝑟 to be meaningful, we investigate what types of information within a set of hand-scripted explanations are most helpful to non-experts for failure and solution identification. Additionally, we investigate how such explanations can be autonomously generated, extending an existing encoder-decoder model, and generalized across environments. We investigate such questions in the context of a robot performing a pick-and-place manipulation task in the home environment. Our results show that explanations capturing the context of a failure and history of past actions, are the most effective for failure and solution identification among non-experts. Furthermore, through a second user evaluation, we verify that our model-generated explanations can generalize to an unseen office environment, and are just as effective as the hand-scripted explanations. CCS CONCEPTS• Human-centered computing → Interaction paradigms.

Correct Me If I'm Wrong: Using Non-Experts to Repair Reinforcement Learning Policies

“…We implemented a proof-of-concept using tabular Qlearning, but we want to test our approach with more complex learning algorithms and policies outside RL. Finally, our approach can complement work on failure explainability [43], e.g., to use OAs for explaining failures to users.…”

Explainable Autonomous Robots: A Survey and Perspective

“…and by presenting the language connected to this in advance. Ehsan et al [69] and Das et al [72] proposed a framework for directly generating linguistic explanations from agent state sequences using an encoder-decoder model.…”

Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study

“…Other XAIP user studies use one domain(Chakraborti et al 2019b;Chakraborti and Kambhampati 2019;Sreedharan et al 2019aLindsay et al 2020;Das, Banerjee, and Chernova 2021), two domains(Sreedharan et al 2019b;Sreedharan, Srivastava, and Kambhampati 2020) or four domains(Krarup et al 2021). …”