Modeling Trust Dynamics in Human-robot Teaming: A Bayesian Inference Approach

Guo, Yaohui; Zhang, Chongjie; Yang, X. Jessie

doi:10.1145/3334480.3383007

Cited by 11 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, there are computational models to infer trust based on human behavior. Examples of this include the Online Probabilistic Trust Inference Model (OPTIMo) [26] and its extensions [9,23]. OPTIMo is one of the pioneers in this area in which they capture trust as a latent variable in a dynamic Bayesian network.…”

Section: Related Workmentioning

confidence: 99%

A Mental Model Based Theory of Trust

Abbaszadeh¹,

Sreedharan²,

Kambhampati³

2023

Preprint

View full text Add to dashboard Cite

Handling trust is one of the core requirements for facilitating effective interaction between the human and the AI agent. Thus, any decision-making framework designed to work with humans must possess the ability to estimate and leverage human trust. In this paper, we propose a mental model based theory of trust that not only can be used to infer trust, thus providing an alternative to psychological or behavioral trust inference methods, but also can be used as a foundation for any trust-aware decision-making frameworks. First, we introduce what trust means according to our theory and then use the theory to define trust evolution, human reliance and decision making, and a formalization of the appropriate level of trust in the agent. Using human subject studies, we compare our theory against one of the most common trust scales (Muir scale) to evaluate 1) whether the observations from the human studies match our proposed theory and 2) what aspects of trust are more aligned with our proposed theory.

show abstract

Section: Related Workmentioning

confidence: 99%

A Mental Model Based Theory of Trust

Abbaszadeh¹,

Sreedharan²,

Kambhampati³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent studies also suggest that humans follow the principles of Bayesian inference when observing the correctness of machine-based decisions. For instance, Wang et al (2018) and Guo et al (2020) analyze in an experimental set-up how observers dynamically update their trust in the machine as they observe the failures and successes of its predictions (without overriding the machine, as in the benchmark of Section 4). These studies find that assuming Bayesian observers can explain the empirical level of human trust in the machine over time.…”

Section: Literature Reviewmentioning

confidence: 99%

Is Your Machine Better Than You? You May Never Know

Véricourt

Gürkan

2023

Management Science

View full text Add to dashboard Cite

Artificial intelligence systems are increasingly demonstrating their capacity to make better predictions than human experts. Yet recent studies suggest that professionals sometimes doubt the quality of these systems and overrule machine-based prescriptions. This paper explores the extent to which a decision maker (DM) supervising a machine to make high-stakes decisions can properly assess whether the machine produces better recommendations. To that end, we study a setup in which a machine performs repeated decision tasks (e.g., whether to perform a biopsy) under the DM’s supervision. Because stakes are high, the DM primarily focuses on making the best choice for the task at hand. Nonetheless, as the DM observes the correctness of the machine’s prescriptions across tasks, the DM updates the DM’s belief about the machine. However, the DM is subject to a so-called verification bias such that the DM verifies the machine’s correctness and updates the DM’s belief accordingly only if the DM ultimately decides to act on the task. In this setup, we characterize the evolution of the DM’s belief and overruling decisions over time. We identify situations under which the DM hesitates forever whether the machine is better; that is, the DM never fully ignores but regularly overrules it. Moreover, the DM sometimes wrongly believes with positive probability that the machine is better. We fully characterize the conditions under which these learning failures occur and explore how mistrusting the machine affects them. These findings provide a novel explanation for human–machine complementarity and suggest guidelines on the decision to fully adopt or reject a machine. This paper was accepted by Elena Katok, special issue on the human–algorithm connection. Supplemental Material: The online appendix is available at https://doi.org/10.1287/mnsc.2023.4791 .

show abstract

“…OPTIMo uses a technique for estimating trust in real-time that depends on the robot's task performance, human intervention, and trust feedback (Xu and Dudek 2015). Trust inference model based on Bayesian inference with Beta-distribution to capture both positive and negative attitude on robot's performance (Guo, Zhang, and Yang 2020) contributes an important extension to OPTIMo. Also, this Bayesian reasoning for trust inference has been considered non-parametrically with Gaussian processes, Recurrent Neural Network (RNN), and a hybrid approach in which trust is a task-dependent latent function (Soh et al 2020).…”

Section: Related Workmentioning

confidence: 99%

Trust-Aware Planning: Modeling Trust Evolution in Iterated Human-Robot Interaction

Abbaszadeh¹,

Verma²,

Sreedharan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Trust between team members is an essential requirement for any successful cooperation. Thus, engendering and maintaining the fellow team members' trust becomes a central responsibility for any member trying to not only successfully participate in the task but to ensure the team achieves its goals. The problem of trust management is particularly challenging in mixed human-robot teams where the human and the robot may have different models about the task at hand and thus may have different expectations regarding the current course of action and forcing the robot to focus on the costly explicable behavior. We propose a computational model for capturing and modulating trust in such longitudinal human-robot interaction, where the human adopts a supervisory role. In our model, the robot integrates human's trust and their expectations from the robot into its planning process to build and maintain trust over the interaction horizon. By establishing the required level of trust, the robot can focus on maximizing the team goal by eschewing explicit explanatory or explicable behavior without worrying about the human supervisor monitoring and intervening to stop behaviors they may not necessarily understand. We model this reasoning about trust levels as a meta reasoning process over individual planning tasks. We additionally validate our model through a human subject experiment.

show abstract

Modeling Trust Dynamics in Human-robot Teaming: A Bayesian Inference Approach

Cited by 11 publications

References 36 publications

A Mental Model Based Theory of Trust

A Mental Model Based Theory of Trust

Is Your Machine Better Than You? You May Never Know

Trust-Aware Planning: Modeling Trust Evolution in Iterated Human-Robot Interaction

Contact Info

Product

Resources

About