Are Well-Calibrated Users Effective Users? Associations Between Calibration of Trust and Performance on an Automation-Aided Task

Merritt, Stephanie M.; Lee, Dong Hoon; Unnerstall, Jennifer L.; Huber, Kelli

doi:10.1177/0018720814561675

Cited by 61 publications

(35 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, a popular task to examine this trust in automation cycle is the bag-screening task. In this task, participants can screen for dangerous objects themselves and follow an automated agent’s recommendation ( Madhavan et al, 2006 ; Madhavan and Wiegmann, 2007a ; Merritt and Ilgen, 2008 ; Merritt et al, 2013 , 2014, 2015 ; Pop et al, 2015 ). A complete neural explanatory mechanism of trust calibration in automated agents would need to include the ERP profile of the observation of the automation’s performance, the evaluation of the automated agent’s decision recommendation as well as the feedback on the consequence of either complying or not complying with the advice of the agent.…”

Section: Discussionmentioning

confidence: 99%

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

Visser

Beatty

Estepp

et al. 2018

Front. Hum. Neurosci.

View full text Add to dashboard Cite

With the rise of increasingly complex artificial intelligence (AI), there is a need to design new methods to monitor AI in a transparent, human-aware manner. Decades of research have demonstrated that people, who are not aware of the exact performance levels of automated algorithms, often experience a mismatch in expectations. Consequently, they will often provide either too little or too much trust in an algorithm. Detecting such a mismatch in expectations, or trust calibration, remains a fundamental challenge in research investigating the use of automation. Due to the context-dependent nature of trust, universal measures of trust have not been established. Trust is a difficult construct to investigate because even the act of reflecting on how much a person trusts a certain agent can change the perception of that agent. We hypothesized that electroencephalograms (EEGs) would be able to provide such a universal index of trust without the need of self-report. In this work, EEGs were recorded for 21 participants (mean age = 22.1; 13 females) while they observed a series of algorithms perform a modified version of a flanker task. Each algorithm’s degree of credibility and reliability were manipulated. We hypothesized that neural markers of action monitoring, such as the observational error-related negativity (oERN) and observational error positivity (oPe), are potential candidates for monitoring computer algorithm performance. Our findings demonstrate that (1) it is possible to reliably elicit both the oERN and oPe while participants monitored these computer algorithms, (2) the oPe, as opposed to the oERN, significantly distinguished between high and low reliability algorithms, and (3) the oPe significantly correlated with subjective measures of trust. This work provides the first evidence for the utility of neural correlates of error monitoring for examining trust in computer algorithms.

show abstract

Section: Discussionmentioning

confidence: 99%

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

Visser

Beatty

Estepp

et al. 2018

Front. Hum. Neurosci.

View full text Add to dashboard Cite

show abstract

“…These measures were selected because of their potential relevance to responses to automation and to ensure that both groups did not differ on personality measures. Research suggests that personality and other individual differences can have strong effects on a number of automation-related outcomes (Merritt, Heimbaugh, LaChapell, & Lee, 2013; Merritt, Lee, Unnerstall, & Huber, 2015; Merritt & Ilgen, 2008; Szalma & Taylor, 2011; Szalma, 2009). …”

Section: Methodsmentioning

confidence: 99%

A Little Anthropomorphism Goes a Long Way

et al. 2017

View full text Add to dashboard Cite

Objective We investigated the effects of exogenous oxytocin on trust, compliance, and team decision making with agents varying in anthropomorphism (computer, avatar, human) and reliability (100%, 50%). Background Recent work has explored psychological similarities in how we trust human-like automation compared to how we trust other humans. Exogenous administration of oxytocin, a neuropeptide associated with trust among humans, offers a unique opportunity to probe the anthropomorphism continuum of automation to infer when agents are trusted like another human or merely a machine. Method Eighty-four healthy male participants collaborated with automated agents varying in anthropomorphism that provided recommendations in a pattern recognition task. Results Under placebo, participants exhibited less trust and compliance with automated aids as the anthropomorphism of those aids increased. Under oxytocin, participants interacted with aids on the extremes of the anthropomorphism continuum similarly to placebos, but increased their trust, compliance, and performance with the avatar, an agent on the midpoint of the anthropomorphism continuum. Conclusion This study provided the first evidence that administration of exogenous oxytocin affected trust, compliance, and team decision making with automated agents. These effects provide support for the premise that oxytocin increases affinity for social stimuli in automated aids. Application Designing automation to mimic basic human characteristics is sufficient to elicit behavioral trust outcomes that are driven by neurological processes typically observed in human-human interactions. Designers of automated systems should consider the task, the individual, and the level of anthropomorphism to achieve the desired outcome.

show abstract

“…Trust is difficult to measure, monitor [21] and especially hard to assess in a real-time manner, as it is often too disruptive to interrupt and ask users to report trust ratings during the course of an interaction. Measuring and monitoring trust, however, is paramount to the success of human-agent teaming [31]. When trust in agents is too high, users tend to have a more complacent attitude, whereas when trust is too low, users tend to overlook or ignore agents' inputs.…”

Section: Trust In Automationmentioning

confidence: 99%

Impact of Agent Reliability and Predictability on Trust in Real Time Human-Agent Collaboration

Daronnat

Azzopardi

Halvey

et al. 2020

Proceedings of the 8th International Conference on Human-Agent Interaction

View full text Add to dashboard Cite

Trust is a prerequisite for effective human-agent collaboration. While past work has studied how trust relates to an agent's reliability, it has been mainly carried out in turn based scenarios, rather than during real-time ones. Previous research identified the performance of an agent as a key factor influencing trust. In this work, we posit that an agent's predictability also plays an important role in the trust relationship, which may be observed based on users' interactions. We designed a 2x2 within-groups experiment with two baseline conditions: (1) no agent (users' individual performance), and (2) near-flawless agent (upper bound). Participants took part in an interactive aiming task where they had to collaborate with different agents that varied in terms of their predictability, and were controlled in terms of their performance. Our results show that agents whose behaviours are easier to predict have a more positive impact on task performance, reliance and trust while reducing cognitive workload. In addition, we modelled the human-agent trust relationship and demonstrated that it is possible to reliably predict users' trust ratings using real-time interaction data. This work seeks to pave the way for the development of trust-aware agents capable of adapting and responding more appropriately to users. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI; Collaborative interaction; User studies; Laboratory experiments.

show abstract

Are Well-Calibrated Users Effective Users? Associations Between Calibration of Trust and Performance on an Automation-Aided Task

Abstract: Users who were better able to perform the task unaided were better able to identify and correct aid failure, suggesting that user task training and expertise may benefit human-automation performance.

Cited by 61 publications

References 22 publications

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

A Little Anthropomorphism Goes a Long Way

Impact of Agent Reliability and Predictability on Trust in Real Time Human-Agent Collaboration

Contact Info

Product

Resources

About