How Accurate Does It Feel? – Human Perception of Different Types of Classification Mistakes

Abstract

Supervised machine learning utilizes large datasets, often with ground truth labels annotated by humans. While some data points are easy to classify, others are hard to classify, which reduces the inter-annotator agreement. This causes noise for the classifier and might affect the user's perception of the classifier's performance. In our research, we investigated whether the classification difficulty of a data point influences how strongly a prediction mistake reduces the "perceived accuracy". In an experimental online study, 225 participants interacted with three fictive classifiers with equal accuracy (73%). The classifiers made prediction mistakes on three different types of data points (easy, difficult, impossible). After the interaction, participants judged the classifier's accuracy. We found that not all prediction mistakes reduced the perceived accuracy equally. Furthermore, the perceived accuracy differed significantly from the calculated accuracy. To conclude, accuracy and related measures seem unsuitable to represent how users perceive the performance of classifiers. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; Human computer interaction (HCI).

One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations

“…(e) After answering all comprehension questions, participants responded to a post-task questionnaire where they reported if they noticed any inconsistencies and rated their confidence in their answers and their perceived AI capacity on a 7-point Likert scale. We adopted single-item measures since they have been shown to effectively assess perceived AI capacity [52,55]. They were also asked to explain the reason for their self-reported ratings.…”

How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)

“…This double use of the term "trustworthiness" is reflected in seminal research on trust [70,79] that uses the term "trustworthiness" to refer to the subjective perception and the objective attributes of the trustee. This distinction is also reflected in the broader trust literature [e.g., 35,53,55,60,73,90,105,110,111]. To clearly distinguish between those two uses of the term trustworthiness, we thus propose that on the side of the trustee, there exists an actual trustworthiness (AT) reflecting the characteristics of the system.…”

“…In the conceptualization of AT and PT, those examples boil down to the fact that the AT of a system was either higher or lower than the trustor's PT of this system [55,60]. Further evidence supporting the importance of understanding the process through which AT is translated into PT comes from research showing that assessing seemingly objective system characteristics such as predictive accuracy translates to strongly varying perceptions of system accuracy for different observers [28,76,90,97].…”

“…Research provided evidence for miscalibration due to a wrongful weighting of cues. For instance, accuracy perceptions were influenced by the focus on precision vs. recall [60] and by the perceived difficulty of the system's task [90]. Furthermore, lay peoples' trust in a system seems to be more heavily influenced by a small sample of personal experiences in their interaction with a system than by accuracy information stemming from a report of accuracy in a large data set [23,95].…”

“…Even if A and B apply the same standards, they may arrive at their judgment in different ways: Whereas A might assess system accuracy by exploring test data sets, for B it may be enough to rely on certification seals to let them expect the system to be accurate enough [29,47,59]. Although both trustors apply the same standards of trustworthiness and even though the system's characteristics remain the same, those trustors might end up at different perceptions of the system's accuracy leading to variation in perceptions of system trustworthiness [55,90].…”