Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracy

Abstract

Mixed-initiative systems allow users to interactively provide feedback to potentially improve system performance. Human feedback can correct model errors and update model parameters to dynamically adapt to changing data. Additionally, many users desire the ability to have a greater level of control and fix perceived flaws in systems they rely on. However, how the ability to provide feedback to autonomous systems influences user trust is a largely unexplored area of research. Our research investigates how the act of providing feedback can affect user understanding of an intelligent system and its accuracy. We present a controlled experiment using a simulated object detection system with image data to study the effects of interactive feedback collection on user impressions. The results show that providing human-in-the-loop feedback lowered both participants’ trust in the system and their perception of system accuracy, regardless of whether the system accuracy improved in response to their feedback. These results highlight the importance of considering the effects of allowing end-user feedback on user trust when designing intelligent systems.

The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems

“…On the contrary, mistrust and distrust can cause users to underestimate the system and rely on themselves, eventually causing them to stop using the automated system in the future. For example, getting people to provide feedback to an intelligent system to fix errors can amplify user mistrust in the system (Honeycutt, Nourani, and Ragan 2020). More related to this paper, Nourani et al (2020b) found that after observing system's weakly-justified predictions, users tend to disagree with the system even when it is right.…”

Leveraging explanations in interactive machine learning: An overview

“…Understanding also plays a role. Failure to understand an explanation may drive users to immediately and unjustifiably distrust the system (Honeycutt et al, 2020) and, rather surprisingly, in some cases the mere fact of being exposed to the machine's internal reasoning may induce a loss of trust (Honeycutt et al, 2020). More generally, the link between interaction and trust is under-explored.…”

Enriching Artificial Intelligence Explanations with Knowledge Fragments

“…This way, the human can act as a teacher to Artificial Intelligence models and learn from them through Explainable Artificial Intelligence. This two-way relationship can lead to a trusted collaboration [29,30].…”