Adaptive trust calibration for human-AI collaboration

Abstract

Safety and efficiency of human-AI collaboration often depend on how humans could appropriately calibrate their trust towards the AI agents. Over-trusting the autonomous system sometimes causes serious safety issues. Although many studies focused on the importance of system transparency in keeping proper trust calibration, the research in detecting and mitigating improper trust calibration remains very limited. To fill these research gaps, we propose a method of adaptive trust calibration that consists of a framework for detecting the inappropriate calibration status by monitoring the user's reliance behavior and cognitive cues called "trust calibration cues" to prompt the user to reinitiate trust calibration. We evaluated our framework and four types of trust calibration cues in an online experiment using a drone simulator. A total of 116 participants performed pothole inspection tasks by using the drone's automatic inspection, the reliability of which could fluctuate depending upon the weather conditions. The participants needed to decide whether to rely on automatic inspection or to do the inspection manually. The results showed that adaptively presenting simple cues could significantly promote trust calibration during over-trust.

Trust in Robots: Challenges and Opportunities

“…This has led to the proliferation of qualitatively different ways to define trust. For instance, trust has been thought of as a belief [13], an attitude [13], an affective response [22], a sense of willingness [23], a form of mutual understanding [24], and as an act of reliance [25].…”

“…Heuristics have been proposed to tackle two different situations: to combat overtrust and to repair trust. As an example of the former, [25] proposed to use visual prompts to nudge users to reevaluate their trust in the robotic system when the user has left the automated system running unattended for too long.…”

Empirical Evaluations of Framework for Adaptive Trust Calibration in Human-AI Cooperation

“…We approach the research challenges with an emphasis on two important aspects of trust in human-AI cooperation: performance and human behavior. We previously proposed a method of adaptive trust calibration [21], using a formal definition of overtrust and under-trust, and conducted an initial evaluation with an over-trust scenario. In the current study, we extend the original method by introducing a third actor called "trust calibration AI" (TCAI) to human-AI cooperation.…”

Feeling of Safety and Comfort Towards a Socially Assistive Unmanned Aerial Vehicle That Monitors People in a Virtual Home

“…In this sense, the concept of trust is very important in the adoption of technologies to assist older adults at home [7,8]. Trust can be defined as an attitudinal judgement of the degree to which a user (the ageing adult) can rely on an agent (the social assistive robot) to achieve its goals under conditions of uncertainty [9]. People are more reluctant to engage with robots if negative consequences are more likely, and once confidence has been lost, people take longer to use this technology again [6,10].…”

“…People are more reluctant to engage with robots if negative consequences are more likely, and once confidence has been lost, people take longer to use this technology again [6,10]. Moreover, safety and efficiency of HRI collaboration often depend on appropriately calibrating trust towards the robot [9] and using a user-centred approach to realise what impacts the development of trust [11]. To date, trust regarding older adults' adoption of assistive technology has been determined in several ways, including whether the elderly feels safe and comfortable with the proposed solution [12][13][14][15].…”