Improving Robot Controller Transparency Through Autonomous Policy Explanation

Abstract

Shared expectations and mutual understanding are critical facets of teamwork. Achieving these in human-robot collaborative contexts can be especially challenging, as humans and robots are unlikely to share a common language to convey intentions, plans, or justifications. Even in cases where human co-workers can inspect a robot's control code, and particularly when statistical methods are used to encode control policies, there is no guarantee that meaningful insights into a robot's behavior can be derived or that a human will be able to efficiently isolate the behaviors relevant to the interaction. We present a series of algorithms and an accompanying system that enables robots to autonomously synthesize policy descriptions and respond to both general and targeted queries by human collaborators. We demonstrate applicability to a variety of robot controller types including those that utilize conditional logic, tabular reinforcement learning, and deep reinforcement learning, synthesizing informative policy descriptions for collaborators and facilitating fault diagnosis by non-experts.

“Dave...I can assure you ...that it’s going to be all right ...” A Definition, Case for, and Survey of Algorithmic Assurances in Human-Autonomy Trust Relationships

“…Some of the answers to these questions lie more in the realm of cognitive science. Still, natural language and other communication modalities could be used [58]. Speci cally, Olah et al [104] investigate how predictions of NNs can be explained through visualizing how di erent parts of the network respond to certain images.…”

Explanation in artificial intelligence: Insights from the social sciences

“…This resurgence is driven by evidence that many AI applications have limited take up, or are not appropriated at all, due to ethical concerns [2] and a lack of trust on behalf of their users [166,101]. The running hypothesis is that by building more transparent, interpretable, or explainable systems, users will be better equipped to understand and therefore trust the intelligent agents [129,25,65].…”

Estimating Attentional Set-Shifting Dynamics in Varying Contextual Bandits

“…This is usually a fundamental step toward selecting the optimal action in both cases of cooperation or competition. Another application is human and artificial agents behavioral modeling which is required for both successful human-AI collaboration and transparent (explainable) Artificial Intelligence (AI) [17,18,19,20,21].…”