Enabling robots to communicate their objectives

Huang, Sandy H.; Held, David; Abbeel, Pieter; Dragan, Anca D.

doi:10.1007/s10514-018-9771-0

Cited by 85 publications

(63 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following [Huang et al, 2017], we formalize the problem of extracting a summary of an agent's behavior as an instance of machine teaching, which aims to find a set of training examples that induces a known target model to learn a pre-specified source model [Zhu, 2015]. In our setting, the agent's policy is the source model that we want to induce, and the target models that we consider are hypotheses about how humans generalize from examples of an agent's behavior-IRL and IL.…”

Section: Methodsmentioning

confidence: 99%

“…One approach relies on heuristics based on agent Q-values and state similarity, to extract diverse, important states Huang et al, 2018]. A second approach formalizes the problem through machine teaching, where the goal is to extract state-action pairs useful for recovering the agent's reward function with IRL [Huang et al, 2017]. Our work extends the latter approach by additionally considering IL.…”

Section: Related Workmentioning

confidence: 99%

“…A recent complementary line of work focuses on summarizing the global behavior of the agent by demonstrating actions taken by the agent in different states . Such summaries have been shown to improve people's * Equal contribution ability to assess agents' capabilities Huang et al, 2018], anticipate agents' actions [Huang et al, 2017] and facilitate trust [Huang et al, 2018].…”

Section: Introductionmentioning

confidence: 99%

“…The first approach applies heuristics related to the diversity or importance of states to determine what is shown in the summary Huang et al, 2018]. The second approach assumes a computational model of how humans will generalize from the summaries provided, and uses that model to optimize the summary for reconstructing the agent's full policy [Huang et al, 2017]. Specifically, [Huang et al, 2017] assumed that people would deploy inverse reinforcement learning (IRL) to infer the agent's reward function from the summary; their summaries were created to perform well given this assumption of human computation.…”

Section: Introductionmentioning

confidence: 99%

“…The second approach assumes a computational model of how humans will generalize from the summaries provided, and uses that model to optimize the summary for reconstructing the agent's full policy [Huang et al, 2017]. Specifically, [Huang et al, 2017] assumed that people would deploy inverse reinforcement learning (IRL) to infer the agent's reward function from the summary; their summaries were created to perform well given this assumption of human computation. The cognitive science literature provides evidence that people sometimes do build models of others' behavior in this way [Baker et al, 2009;Baker et al, 2011], making IRL a plausible model.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Exploring Computational User Models for Agent Policy Summarization

Lage

Lifschitz

Doshi‐Velez

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

AI agents are being developed to support high stakes decision-making processes from driving cars to prescribing drugs, making it increasingly important for human users to understand their behavior. Policy summarization methods aim to convey strengths and weaknesses of such agents by demonstrating their behavior in a subset of informative states. Some policy summarization methods extract a summary that optimizes the ability to reconstruct the agent's policy under the assumption that users will deploy inverse reinforcement learning. In this paper, we explore the use of different models for extracting summaries. We introduce an imitation learning-based approach to policy summarization; we demonstrate through computational simulations that a mismatch between the model used to extract a summary and the model used to reconstruct the policy results in worse reconstruction quality; and we demonstrate through a human-subject study that people use different models to reconstruct policies in different contexts, and that matching the summary extraction model to these can improve performance. Together, our results suggest that it is important to carefully consider user models in policy summarization.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Exploring Computational User Models for Agent Policy Summarization

Lage

Lifschitz

Doshi‐Velez

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

Patching interpretable And‐Or‐Graph knowledge representation using augmented reality

Liu

Zhu

2021

Applied AI Letters

View full text Add to dashboard Cite

We present a novel augmented reality (AR) interface to provide effective means to diagnose a robot's erroneous behaviors, endow it with new skills, and patch its knowledge structure represented by an And-Or-Graph (AOG). Specifically, an AOG representation of opening medicine bottles is learned from human demonstration and yields a hierarchical structure that captures the spatiotemporal compositional nature of the given task, which is highly interpretable for the users. Through a series of psychological experiments, we demonstrate that the explanations of a robotic system, inherited from and produced by the AOG, can better foster human trust compared to other forms of explanations. Moreover, by visualizing the knowledge structure and robot states, the AR interface allows human users to intuitively understand what the robot knows, supervise the robot's task planner, and interactively teach the robot with new actions. Together, users can quickly identify the reasons for failures and conveniently patch the current knowledge structure to prevent future errors. This capability demonstrates the interpretability of our knowledge representation and the new forms of interactions afforded by the proposed AR interface.augmented reality (AR), explainable artificial intelligence (XAI), robot learning | INTRODUCTIONThe ever-growing vast amount of data and rapid-increasing computing power have recently boosted a data-driven machine learning paradigm. Despite promising progress in model performance, such pure data-driven approaches, especially methods using deep neural networks, have one well-known limitation-the lack of interpretability. Various attempts, therefore, have been made to alleviate this shortcoming, such as visualizing filter responses, 1-3 developing communication protocols, [4][5][6][7] and generating text descriptions for images, 8,9 or robot behaviors. 10,11 However, these explanation mechanisms only suffice with two-dimensional (2D) interactions (ie, using computer screens) and fall short of interacting with physical robots, especially in some mission-critical settings that require supervising multiple robots. Therefore, an interpretable knowledge representation for robots and an effective explanation interface beyond 2D are needed for better situational awareness, richer spatial information, and in situ explanations during human-robot interaction. Of note, less attention has been paid, for physical robots, in introducing

show abstract