The Effect of Automation Reliability on User Automation Trust and Reliance in a Search-and-Rescue Scenario

Abstract

Advances in modern day technology are rapidly increasing the ability of engineers to automate ever more complicated tasks. Often these automated aids are paired with human operators who can supervise their work to ensure that it is free of errors and to even take control of the system if it malfunctions (e.g., pilots supervising an autopilot feature). The goal of this collaboration, between humans and machines, is that it can enhance performance beyond what would be possible by either alone. Arguably the success of this partnership depends in part upon attributions an operator develops that help guide their interaction with the automation. One particular factor that has been shown to guide operator reliance on an automated 'teammate' is trust. The following study examined 140 participants performing a simulated search-and-rescue task. The goal of this experiment was to examine the relationship between automated agent's reliability, operator trust, operator reliance, and performance scores. Results indicated that greater automation reliability is positively correlated with greater user reliance (r=.66), perceived trust (r=.21), and performance scores (r=.34). These results indicate that more reliable aids are rated as significantly higher in terms of perceived trust and relied upon more than less reliable aids. Additionally, the size of the effect is much larger for operator behaviors (i.e., reliance) compared to more subjective measures (i.e., self-reported trust).

Trust in Automation

“…Several different aspects of performance have been studied. First, numerous studies have shown that the reliability and validity of an automated system's functions are important antecedents of trust (Bailey & Scerbo, 2007;Madhavan & Wiegmann, 2007a;Oduor & Campbell, 2007;Seong & Bisantz, 2008;Ross, 2008;Ross, Szalma, Hancock, Barnett, & Taylor, 2008;Sanchez et al, 2004;Wetzel, 2005). Reliability refers to the consistency of an automated system's functions, and validity refers to the degree to which an automated system performs the intended task.…”

Relationships between User Demographics and User Trust in an Autonomous Agent

“…It is acknowledged that perceived reliability is only one aspect of user trust (Hoff & Bashir, 2015;McCarley, Wiegmann, Wickens, & Kramer, 2003). However, using perceived reliability as a metric to quantify trust has been shown to be an effective means with which to measure how much a user will trust in an automated decision aid (Ross et al, 2008). This research uses this same metric with an autonomous agent rather than just an automated decision aid.…”

“…One aspect of successful teams is trust between teammates. Trust in automation is defined as the human user's confidence that the automated system will help the user achieve his or her goals in environments of uncertainty and vulnerability (Ross et al, 2008). Research has shown that in most environments, performance is higher when the user trusts the automation to perform its role (Endsley, 2015;Ho et al, 2005;.…”

Designing an Automated Agent to Encourage Human Reliance

“…As technologies improved, autonomous systems gained the capability to diagnose failure or recommend operator actions. Ross et. al.…”