Application of a System-Wide Trust Strategy when Supervising Multiple Autonomous Agents

Abstract

When interacting with complex systems, the manner in which an operator trusts automation influences system performance. Recent studies have demonstrated that people tend to apply trust broadly rather than exhibiting specific trust in each component of the system in a calibrated manner (e.g. Keller & Rice, 2010). While this System–Wide Trust effect has been established for basic situations such as judging gauges, it has not been studied in realistic settings such as collaboration with autonomous agents in a multi-agent system. This study utilized a multiple UAV control simulation, to explore how people apply trust in multi autonomous agents in a supervisory control setting. Participants interacted with four UAVs that utilized automated target recognition (ATR) systems to identify targets as enemy or friendly. When one of the autonomous agents was inaccurate and performance information was provided, participants were 1) less accurate, 2) more likely to verify the ATR’s determination, 3) spent more time verifying images, and 4) rated the other systems as less trustworthy even though they were 100% correct. These findings support previous work that demonstrated the prevalence of system-wide trust and expand the conditions in which system-wide trust strategies are applied. This work suggests that multi-agent systems should provide carefully designed cues and training to mitigate the system-wide trust effect.

Not All Information Is Equal: Effects of Disclosing Different Types of Likelihood Information on Trust, Compliance and Reliance, and Task Performance in Human-Automation Teaming

“…At last, consistent with findings from previous studies, our results showed that as the automated threat detector became more reliable, participants' trust in and dependence on the threat detector increased, and their dual-task performance improved (Neyedli et al, 2011;Walliser et al, 2016;Wang et al, 2009). conclusIon Although disclosing likelihood information has been proposed as a design solution to promote proper trust and dependence, and to enhance human-automation team performance, prior studies showed mixed results (Bagheri & Jamieson, 2004;Dzindolet et al, 2002;Fletcher et al, 2017;Walliser et al, 2016;Wang et al, 2009). The goal of this study was to experimentally examine the effects of presenting different types of likelihood information.…”

“…Yet, results from these studies seem to be inconsistent. Some studies revealed that the likelihood information significantly helped human operators calibrate their trust, adjust their reliance and compliance behaviors, and enhance human-automation team performance (McGuirl & Sarter, 2006;Walliser, de Visser, & Shaw, 2016;Wang, Jamieson, & Hollands, 2009). Other studies, however, reported that human operators did not trust or depend on automated decision aids appropriately even when the likelihood information was disclosed (Bagheri & Jamieson, 2004;Fletcher, Bartlett, Cockshell, & McCarley, 2017).…”

“…Results showed that both types of decision aids were rated more favorably when its reliability was disclosed. More recently, Walliser et al (2016) conducted a study where participants interacted with four unmanned aerial vehicles (UAVs) that used automated target recognition (ATR) systems to identify targets as enemy or friendly. Results showed that when participants were informed of the overall success likelihood information ("corrected identification rate" in the article), participants tended to apply a more appropriate strategy when interacting with the automation, resulting in better task performance.…”

Mixing It Up: How Mixed Groups of Humans and Machines Modulate Conformity

“…Furthermore, adding humans to a decision aid may help with acceptance of automated advice when operators have a tendency to reject the recommendations of an accurate automated aid (Cummings, Clare, & Hart, 2010; Dzindolet et al, 2001). Finally, mixing sources of advice within a human-machine team could further help in diminishing system-wide trust , an effect where operators generalize the incorrect advice of one aid to other identical aids (Keller & Rice, 2010; Walliser, de Visser, & Shaw, 2016). Taken together, these suggestions could help improve human-machine team performance using group composition as deliberate design variable to steer group consensus in beneficial directions that help improve group decision making.…”

Designing Artificial Agents as Social Companions

“…The first part of the discussion will focus on the strategies operators employ when interacting with multiple systems. Recent studies have demonstrated that people tend to apply trust broadly rather than exhibiting specific trust in each component of a system in a calibrated manner (Keller & Rice, 2010;Walliser et al, 2016). This can be problematic because this means that a single, unreliable agent or component of a system can create a pull-down effect such that the rate of operator trust in other functionally similar agents or components is lowered (Keller & Rice, 2010).…”