Whoops! Something Went Wrong: Errors, Trust, and Trust Repair Strategies in Human Agent Teaming

Rebensky, Summer; Carmody, Kendall; Ficke, Cherrise; Nguyen, Daniel; Carroll, Meredith; Wildman, Jessica L.; Thayer, Amanda L.

doi:10.1007/978-3-030-77772-2_7

Cited by 6 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there is a considerable body of work on the broader area of trust in automation (e.g., alarms, robotics, and unmanned systems), there is considerably less research on trust in AI more specifically. Trust has been defined by Lee and See (2004, p. 54), as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability.” Trust is not a binary phenomenon, but rather a spectrum with a considerable gray area between trust and distrust (Roff & Danks, 2018), where trust is calibrated over time based on interactions with the system (Schaefer, et al, 2016; Rebensky, et al 2021; Yang, Schemanske et al (2021)). Trust (or, to be more specific, calibrated trust) is important for effective human–AI teaming.…”

Section: Trust In Aimentioning

confidence: 99%

A Naturalistic Investigation of Trust, AI, and Intelligence Work

Dorton

Harper

2022

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

Artificial Intelligence (AI) is often viewed as the means by which the intelligence community will cope with increasing amounts of data. There are challenges in adoption, however, as outputs of such systems may be difficult to trust, for a variety of factors. We conducted a naturalistic study using the Critical Incident Technique (CIT) to identify which factors were present in incidents where trust in an AI technology used in intelligence work (i.e., the collection, processing, analysis, and dissemination of intelligence) was gained or lost. We found that explainability and performance of the AI were the most prominent factors in responses; however, several other factors affected the development of trust. Further, most incidents involved two or more trust factors, demonstrating that trust is a multifaceted phenomenon. We also conducted a broader thematic analysis to identify other trends in the data. We found that trust in AI is often affected by the interaction of other people with the AI (i.e., people who develop it or use its outputs), and that involving end users in the development of the AI also affects trust. We provide an overview of key findings, practical implications for design, and possible future areas for research.

show abstract

Section: Trust In Aimentioning

confidence: 99%

A Naturalistic Investigation of Trust, AI, and Intelligence Work

Dorton

Harper

2022

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

show abstract

“…First, they have conducted research related to the heads-up vs. heads-down display of UAS information to the operator and the impact that has on operator situation awareness and performance (Rebensky et al, 2021). They have also examined the usability of technology such as the use of augmented reality glasses during UAS operation in the field (Rebensky et al, 2023). They have also conducted research in the UAS domain related to human agent teams (HAT) such as examining how various levels of automation impact an operator's trust, workload and performance (Rebensky et al, 2022).…”

Section: Meredith Carroll: Academic Research Supporting Uas Operatorsmentioning

confidence: 99%

From Selection to Design: Diverse Perspectives of Human Factors Issues in Unmanned Aerial Systems

Beadle,

Kaplan,

Braly

et al. 2023

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Unmanned Aerial Systems (UASs) serve many roles in the broader defense space- with mission sets like reconnaissance, remote delivery, and supporting other aircraft operations. In the future of UASs, human factors professionals serve a broad role in advocating for the users in the design process. This discussion panel will involve individuals with different experiences working with UAS technologies and its introduction to the end user. The panel will consist of the following topics: capturing user feedback, control station design for supervisory control, selection of operators as this technology grows, and broad issues in public policy. The panelists with a variety of civilian and military experience will highlight their day-to-day role as well as what they see as upcoming topics worth researching in this domain.

show abstract

“…These findings suggest that trust between teammates (human and machine) is related to team performance. Related works have proposed models and guidelines for implementing trust repair in human-machine teams (de Visser et al, 2020;Rebensky et al, 2021), but call for additional empirical studies to provide validation and better explore the relationship between trust and team effectiveness.…”

Section: Trust In Human-artificial Intelligence Teamingmentioning

confidence: 99%

Exploring the Relationship Between Ethics and Trust in Human–Artificial Intelligence Teaming: A Mixed Methods Approach

Textor

Zhang

Lopez

et al. 2022

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

Advancements and implementations of autonomous systems coincide with an increased concern for the ethical implications resulting from their use. This is increasingly relevant as autonomy fulfills teammate roles in contexts that demand ethical considerations. As AI teammates (ATs) enter these roles, research is needed to explore how an AT’s ethics influences human trust. This current research presents two studies which explore how an AT’s ethical or unethical behavior impacts trust in that teammate. In Study 1, participants responded to scenarios of an AT recommending actions which violated or abided by a set of ethical principles. The results suggest that ethicality perceptions and trust are influenced by ethical violations, but only ethicality depends on the type of ethical violation. Participants in Study 2 completed a focus group interview after performing a team task with a simulated AT that committed ethical violations and attempted to repair trust (apology or denial). The focus group responses suggest that ethical violations worsened perceptions of the AT and decreased trust, but it could still be trusted to perform tasks. The AT’s apologies and denials did not repair damaged trust. The studies’ findings suggest a nuanced relationship between trust and ethics and a need for further investigation into trust repair strategies following ethical violations.

show abstract

Whoops! Something Went Wrong: Errors, Trust, and Trust Repair Strategies in Human Agent Teaming

Cited by 6 publications

References 47 publications

A Naturalistic Investigation of Trust, AI, and Intelligence Work

A Naturalistic Investigation of Trust, AI, and Intelligence Work

From Selection to Design: Diverse Perspectives of Human Factors Issues in Unmanned Aerial Systems

Exploring the Relationship Between Ethics and Trust in Human–Artificial Intelligence Teaming: A Mixed Methods Approach

Contact Info

Product

Resources

About