Sequential Explanations with Mental Model-Based Policies

Yeung, Arnold; Joshi, Shalmali; Williams, Joseph Jay; Rudzicz, Frank

doi:10.48550/arxiv.2007.09028

Cited by 3 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In parallel, significant research in human-AI partnership has considered how to directly optimize the team performance in AI-assisted decision-making Lai et al (2020); Lai and Tan (2019); Bansal et al (2020Bansal et al ( , 2019b; Green and Chen (2019); Okamura and Yamada (2020). The perception of the human agents plays an important role in collaborative decision-making (Yeung et al, 2020;Bansal et al, 2019a;Lee, 2018). These works experiment with several heuristic-driven explanation strategies that only partially take into account the characteristics of the human at the end of the decision-making pipeline.…”

Section: Related Workmentioning

confidence: 99%

Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making

Rastogi¹,

Zhang²,

Wei³

et al. 2020

Preprint

View full text Add to dashboard Cite

Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decisionmakers in AI-assisted decision-making, where humans are the consumers of AI model predictions and the ultimate decision-makers in high-stakes applications. However, people's perception and understanding is often distorted by their cognitive biases, like confirmation bias, anchoring bias, availability bias, to name a few. In this work, we use knowledge from the field of cognitive science to account for cognitive biases in the human-AI collaborative decision-making system and mitigate their negative effects. To this end, we mathematically model cognitive biases and provide a general framework through which researchers and practitioners can understand the interplay between cognitive biases and human-AI accuracy. We then focus on anchoring bias, a bias commonly witnessed in human-AI partnerships. We devise a cognitive science-driven, time-based approach to de-anchoring. A user experiment shows the effectiveness of this approach in human-AI collaborative decision-making. Using the results from this first experiment, we design a time allocation strategy for a resource constrained setting so as to achieve optimal human-AI collaboration under some assumptions. A second user study shows that our time allocation strategy can effectively debias the human when the AI model has low confidence and is incorrect.

show abstract

Section: Related Workmentioning

confidence: 99%

Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making

Rastogi¹,

Zhang²,

Wei³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…However, in interpersonal explanations, it is desirable to change not only the information to be explained but also the manner in which the explanation is presented according to the user. Therefore, Yeung et al [70] proposed a method to select the best available explanation presentation method by incorporating the process of explanation presentation and user understanding into a reinforcement learning framework. In order to apply the method to real-world problems, further discussion on reward design and training data acquisition methods would be beneficial.…”

Section: Verbalization and Visualization Of Explanationsmentioning

confidence: 99%

Explainable Autonomous Robots: A Survey and Perspective

Sakai¹,

Nagai

2021

Preprint

View full text Add to dashboard Cite

Advanced communication protocols are critical to enable the coexistence of autonomous robots with humans. Thus, the development of explanatory capabilities is an urgent first step toward autonomous robots. This survey provides an overview of the various types of "explainability" discussed in machine learning research. Then, we discuss the definition of "explainability" in the context of autonomous robots (i.e., explainable autonomous robots) by exploring the question "what is an explanation?" We further conduct a research survey based on this definition and present some relevant topics for future research.

show abstract

“…Hilgard et al (2019) learn to visualize high-dimensional examples to assist users with one-step classification tasks, whereas we focus on sequential decision-making and make minimal assumptions about the desired task. Yeung et al (2020) use a human-in-the-loop reinforcement learning method to train an agent to sequentially explain black-box model predictions to a human auditor, where the agent is rewarded for causing the user's mental model of the predictive model to match the actual predictive model. Our work differs in that it focuses on improving users' situational awareness in control tasks with partial observations, rather than improving model interpretability.…”

Section: Assistive State Estimationmentioning

confidence: 99%

Assisted Perception: Optimizing Observations to Communicate State

Reddy,

Levine,

Dragan

2020

Preprint

View full text Add to dashboard Cite

We aim to help users estimate the state of the world in tasks like robotic teleoperation and navigation with visual impairments, where users may have systematic biases that lead to suboptimal behavior: they might struggle to process observations from multiple sensors simultaneously, receive delayed observations, or overestimate distances to obstacles. While we cannot directly change the user's internal beliefs or their internal state estimation process, our insight is that we can still assist them by modifying the user's observations. Instead of showing the user their true observations, we synthesize new observations that lead to more accurate internal state estimates when processed by the user. We refer to this method as assistive state estimation (ASE): an automated assistant uses the true observations to infer the state of the world, then generates a modified observation for the user to consume (e.g., through an augmented reality interface), and optimizes the modification to induce the user's new beliefs to match the assistant's current beliefs. To predict the effect of the modified observation on the user's beliefs, ASE learns a model of the user's state estimation process: after each task completion, it searches for a model that would have led to beliefs that explain the user's actions. We evaluate ASE in a user study with 12 participants who each perform four tasks: two tasks with known user biases -bandwidth-limited image classification and a driving video game with observation delay -and two with unknown biases that our method has to learn -guided 2D navigation and a lunar lander teleoperation video game. ASE's general-purpose approach to synthesizing informative observations enables a different assistance strategy to emerge in each domain, such as quickly revealing informative pixels to speed up image classification, using a dynamics model to undo observation delay in driving, identifying nearby landmarks for navigation, and exaggerating a visual indicator of tilt in the lander game. The results show that ASE substantially improves the task performance of users with bandwidth constraints, observation delay, and other unknown biases.Code, data, and videos at https://sites.google.com/berkeley.edu/ase

show abstract

Sequential Explanations with Mental Model-Based Policies

Cited by 3 publications

References 23 publications

Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making

Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making

Explainable Autonomous Robots: A Survey and Perspective

Assisted Perception: Optimizing Observations to Communicate State

Contact Info

Product

Resources

About