Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems

He, Gaole; Lucie, Kuiper,; Gadiraju, Ujwal

doi:10.1145/3544548.3581025

Cited by 27 publications

(5 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Reliance and Its Appropriateness We measure the reliance of participants on the AI system via two metrics [22,50]: User Experience We measured participants' perceived autonomy [13], mental demand [21,30,33], perceived complexity [8], engagement [6], future use [30,31], satisfaction [8,19], perceived helpfulness [7,9,28], trust [48], self-efficacy [23] via 7-point Likert scales. The detailed questions are shown in Table 1.…”

Section: Measurements and Analysis Methodsmentioning

confidence: 99%

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Wang

Yin

2021

26th International Conference on Intelligent User Interfaces

171

View full text Add to dashboard Cite

This paper contributes to the growing literature in empirical evaluation of explainable AI (XAI) methods by presenting a comparison on the effects of a set of established XAI methods in AI-assisted decision making. Specifically, based on our review of previous literature, we highlight three desirable properties that ideal AI explanations should satisfy-improve people's understanding of the AI model, help people recognize the model uncertainty, and support people's calibrated trust in the model. Through randomized controlled experiments, we evaluate whether four types of common model-agnostic explainable AI methods satisfy these properties on two types of decision making tasks where people perceive themselves as having different levels of domain expertise in (i.e., recidivism prediction and forest cover prediction). Our results show that the effects of AI explanations are largely different on decision making tasks where people have varying levels of domain expertise in, and many AI explanations do not satisfy any of the desirable properties for tasks that people have little domain expertise in. Further, for decision making tasks that people are more knowledgeable, feature contribution explanation is shown to satisfy more desiderata of AI explanations, while the explanation that is considered to resemble how human explain decisions (i.e., counterfactual explanation) does not seem to improve calibrated trust. We conclude by discussing the implications of our study for improving the design of XAI methods to better support human decision making. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; • Computing methodologies → Machine learning.

show abstract

Section: Measurements and Analysis Methodsmentioning

confidence: 99%

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Wang

Yin

2021

26th International Conference on Intelligent User Interfaces

171

View full text Add to dashboard Cite

show abstract

“…With a similar idea to measuring quality, we embed error-based metrics including E LLM and AE LLM which compute the error and absolute error between crowd workers and LLM, respectively. In addition, we apply Agreement Fraction (He, Kuiper, and Gadiraju 2023), which calculates the rate of crowd workers' decisions that agree with the LLM's advice. Following previous research (Tolmeijer et al 2022), we consider two validated questionnaires using Likert scales to measure the subjective trust in LLM: Trust in Automation (TiA) (Körber 2019) and Affinity for Technology Interaction Scale (ATI) (Franke, Attig, and Wessel 2019).…”

Section: Methodsmentioning

confidence: 99%

“…In addition, we compose each unit using 4 statements that are correctly labeled by GPT-3.5, while 2 statements were incorrectly labeled to mimic the actual GPT-3.5's accuracy on our dataset. The order of the 6 statements is then shuffled for every crowd worker and every task to remove potential learning effects (He, Kuiper, and Gadiraju 2023).…”

Section: Crowdsourcing Task Designmentioning

confidence: 99%

On the Role of Large Language Models in Crowdsourcing Misinformation Assessment

Xu,

Han,

Sadiq

et al. 2024

ICWSM

View full text Add to dashboard Cite

The proliferation of online misinformation significantly undermines the credibility of web content. Recently, crowd workers have been successfully employed to assess misinformation to address the limited scalability of professional fact-checkers. An alternative approach to crowdsourcing is the use of large language models (LLMs). These models are however also not perfect. In this paper, we investigate the scenario of crowd workers working in collaboration with LLMs to assess misinformation. We perform a study where we ask crowd workers to judge the truthfulness of statements under different conditions: with and without LLMs labels and explanations. Our results show that crowd workers tend to overestimate truthfulness when exposed to LLM-generated information. Crowd workers are misled by wrong LLM labels, but, on the other hand, their self-reported confidence is lower when they make mistakes due to relying on the LLM. We also observe diverse behaviors among crowd workers when the LLM is presented, indicating that leveraging LLMs can be considered a distinct working strategy.

show abstract

“…Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 24.03.2024 und verknüpften Fähigkeiten zu beobachten und kann zu einer Überschätzung der eigenen Fähigkeiten, verbunden mit einer Unterschätzung der Risiken führen (He, Kuiper, und Gadiraju 2023).…”

Section: Pädagogikunclassified

Künstliche Intelligenz im Kontext von Kompetenzen, Prüfungen und Lehr-Lern-Methoden

Klar,

Schleiss

2024

MedienPädagogik

View full text Add to dashboard Cite

Der Beitrag untersucht Auswirkungen und Fragestellungen von generativen Sprachmodellen, einer Form der Künstlichen Intelligenz (KI), in den Bereichen von Kompetenzen, Prüfungen und Lehr-Lern-Szenarien. Wir diskutieren, inwiefern KI neue Kompetenzen erfordert, Kompetenzen an Relevanz verlieren und Kompetenzen verloren gehen. Die teilweise veränderten Kompetenzen wirken auch auf die Prüfungsgestaltung: Es stellt sich die Frage, wo KI aus Prüfungen ausgeschlossen und wo sie bewusst eingeschlossen werden sollte. Beide Entscheidungen haben Einfluss auf die Zuverlässigkeit und Alltagsnähe der Prüfungen. KI wirkt hier als Verstärker bestehender Fragen und Problemstellungen. Im Bereich der Lehr-Lern-Szenarien eröffnet KI Lernenden die Möglichkeit, Lernmedien selbst zu adaptieren. Darüber hinaus kann KI als weiterer Akteur in Lehr-Lern-Settings hinzukommen und es stellen sich Fragen, wie dies die soziale Situation des Lernens verändern kann. Im Sinne des Constructive Alignment argumentiert dieser Beitrag, dass KI nicht isoliert, sondern didaktisch verschränkt in Kompetenzen, Prüfungen und Lehr-Lern-Szenarien integriert werden muss. Für die Diskussion und Exploration der sich daraus ergebenden Fragen brauchen Bildungsinstitutionen Offenheit und Experimentierräume.

show abstract

Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems

Cited by 27 publications

References 62 publications

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

On the Role of Large Language Models in Crowdsourcing Misinformation Assessment

Künstliche Intelligenz im Kontext von Kompetenzen, Prüfungen und Lehr-Lern-Methoden

Contact Info

Product

Resources

About