Trust Development in Military and Civilian Human–Agent Teams: The Effect of Social-Cognitive Recovery Strategies

Abstract

Autonomous agents (AA) will increasingly be deployed as teammates instead of tools. In many operational situations, flawless performance from AA cannot be guaranteed. This may lead to a breach in the human’s trust, which can compromise collaboration. This highlights the importance of thinking about how to deal with error and trust violations when designing AA. The aim of this study was to explore the influence of uncertainty communication and apology on the development of trust in a Human–Agent Team (HAT) when there is a trust violation. Two experimental studies following the same method were performed with (I) a civilian group and (II) a military group of participants. The online task environment resembled a house search in which the participant was accompanied and advised by an AA as their artificial team member. Halfway during the task, an incorrect advice evoked a trust violation. Uncertainty communication was manipulated within-subjects, apology between-subjects. Our results showed that (a) communicating uncertainty led to higher levels of trust in both studies, (b) an incorrect advice by the agent led to a less severe decline in trust when that advice included a measure of uncertainty, and (c) after a trust violation, trust recovered significantly more when the agent offered an apology. The two latter effects were only found in the civilian study. We conclude that tailored agent communication is a key factor in minimizing trust reduction in face of agent failure to maintain effective long-term relationships in HATs. The difference in findings between participant groups emphasizes the importance of considering the (organizational) culture when designing artificial team members.

The effect of social-cognitive recovery strategies on likability, capability and trust in social robots

“…Ahead of the development of any structured model of trust as a social construct for HRI, a range of studies have nonetheless demonstrated user trust can be influenced through robots' simulated social or affective interactions. Examples include using rhetorical persuasion [14], using gesture and expression [15], taking blame [16] or offering apologies for errors [18], [19], and making promises to change behaviours [20]. Emerging social models, drawing from such examples, argue that trust as a social construct, as seen in human-human interaction [1], [3], has relevance in HRI [18], [21].…”

How Space Operations Drive Innovation In Human Healthcare

“…In the long run, AI will enable machines to match or outperform critical aspects of human intelligence, allowing them to adapt to unexpected and changing situations regardless of human intervention. In many operational scenarios, such as the military [15], autonomous intelligent agents and humans are used concurrently to achieve common goals, implying that they are expected to make decisions together. Coordination between agents and humans is found to be more important than providing independent functions in operations [15].…”

“…In many operational scenarios, such as the military [15], autonomous intelligent agents and humans are used concurrently to achieve common goals, implying that they are expected to make decisions together. Coordination between agents and humans is found to be more important than providing independent functions in operations [15]. Agents work in uncertain areas when information is lacking, and errors undermine human trust.…”

“…The human component of autonomous intelligent agents is critical to establishing a longterm collaborative relationship between humans and technology. Humans and agents become interdependent when agents are used as teammates rather than tools, and tailored agent communication determines the evolution of human trust in intelligent agents [15].…”

Using a Virtual Reality House-Search Task to Measure Trust During Human-Agent Interaction (Demo Paper)

“…The two buildings are designed to be similar, but include different details. While searching the buildings, participants are accompanied by a drone that tells them whether it detects threats in the environment by advising them to either move carefully or to proceed normally via automated audio messages (see [5]). Level of trust in the drone is repeatedly measured using a virtual slider in the VR environment [4], [6].…”