Can You Trust Your Trust Measure?

Abstract

Trust in human-robot interactions (HRI) is measured in two main ways: through subjective questionnaires and through behavioral tasks. To optimize measurements of trust through questionnaires, the field of HRI faces two challenges: the development of standardized measures that apply to a variety of robots with different capabilities, and the exploration of social and relational dimensions of trust in robots (e.g., benevolence). In this paper we look at how different trust questionnaires [18,30,35] fare given these challenges that pull in different directions (being general vs. being exploratory) by studying whether people think the items in these questionnaires are applicable to different kinds of robots and interactions. In Study 1 we show that after being presented with a robot (non-humanoid) and an interaction scenario (fire evacuation), participants rated multiple questionnaire items such as "This robot is principled" as "Non-applicable to robots in general" or "Non-applicable to this robot. " In Study 2 we show that the frequency of these ratings change (indeed, even for items rated as N/A to robots in general) when a new scenario is presented (game playing with a humanoid robot). Finally, while overall trust scores remained robust to N/A ratings, our results revealed potential fallacies in the way these scores are commonly interpreted. We conclude with recommendations for the development, use and results-reporting of trust questionnaires for future studies, as well as theoretical implications for the field of HRI. CCS CONCEPTS• Human-centered computing → Interaction paradigms; • Computer systems organization → Robotics.

Measurement of Trust in Automation: A Narrative Review and Reference Guide

“…Effort should be undertaken to understand whether popular trust questionnaires are equally applicable and universal or whether the results are dependent on the nature of the agent in question, independent from trust. Some initial efforts have been made to explore the efficacy of different trust questionnaires in different scenarios (e.g., Chita-Tegmark et al, 2021 ) – sufficient evaluation of available measures could enable the creation of an objectively based flowchart for selecting trust measures based on agent and task criteria.…”

No effect of the smell of hexanal on trust in human-robot interaction

“…Here, we assess trust on two levels by measuring trial-by-trial trust decisions during a collaborative perceptual decision-making task and general trust as measured by an established questionnaire. These measures have successfully been used in previous human-robot studies [e.g., 28,29,32], and were found to correlate moderately in the present study. However, we did not observe an effect of hexanal on these measures.…”

“…Finally, cross-platform generalisability [48,49] could play a role. In a recent evaluation of trust questionnaires most often employed in human-robot interaction, issues of generalisability have been reported [32]. Items used in a questionnaire not necessarily generalise across robots or situations.…”

“…Lyons and Guznov specifically adjusted the RIS to assess the level of trust in robots by measuring the anticipated, trust-related attitudes and behaviours towards the robot. The RIS was chosen to assess trust in the robot because of the results in a study by Chita-Tegmark et al [32]. In this study [32], four questionnaires that measure trust in human-robot interaction were compared.…”

“…The RIS was chosen to assess trust in the robot because of the results in a study by Chita-Tegmark et al [32]. In this study [32], four questionnaires that measure trust in human-robot interaction were compared. The RIS scored best, as no items were labelled "Not applicable to robots in general".…”