Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Lai, Vivian; Chen, Chacha; Liao, Q. Vera; Smith-Renner, Alison; Tan, Chenhao

doi:10.48550/arxiv.2112.11471

Cited by 43 publications

(67 citation statements)

References 73 publications

(424 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work in human-AI decision-making has started to discuss AR in the context of AI advice. Lai et al [14] gives an overview of empirical studies that analyze AI advice considering AR. For example, Chandrasekaran et al [4] analyze whether humans can learn to predict the model behavior.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Schemmer¹,

Hemmer²,

Kühl³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many important decisions in daily life are made with the help of advisors, e.g., decisions about medical treatments or financial investments. Whereas in the past, advice has often been received from human experts, friends, or family, advisors based on artificial intelligence (AI) have become more and more present nowadays. Typically, the advice generated by AI is judged by a human and either deemed reliable or rejected. However, recent work has shown that AI advice is not always beneficial, as humans have shown to be unable to ignore incorrect AI advice, essentially representing an over-reliance on AI. Therefore, the aspired goal should be to enable humans not to rely on AI advice blindly but rather to distinguish its quality and act upon it to make better decisions. Specifically, that means that humans should rely on the AI in the presence of correct advice and self-rely when confronted with incorrect advice, i.e., establish appropriate reliance (AR) on AI advice on a case-by-case basis. Current research lacks a metric for AR. This prevents a rigorous evaluation of factors impacting AR and hinders further development of human-AI decision-making. Therefore, based on the literature, we derive a measurement concept of AR. We propose to view AR as a two-dimensional construct that measures the ability to discriminate advice quality and behave accordingly. In this article, we derive the measurement concept, illustrate its application and outline potential future research.CCS Concepts: • Human-centered computing → HCI theory, concepts and models; Empirical studies in HCI; • Computing methodologies → Artificial intelligence.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For illustration, we focus on the explainability of AI advice as a design decision. XAI is intensively discussed in research with regards to its impact on human-AI decision-making in general and AR in specific [1,2,14,29].…”

Section: Illustration Of Appropriate Reliance On Ai Advicementioning

confidence: 99%

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Schemmer¹,

Hemmer²,

Kühl³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Building on a recent survey [32], we identify 30 papers that: 1) use machine learning models and explanations with the goal of improving human understanding; and 2) conduct empirical human studies to evaluate human understanding with quantitative metrics. Although human-subject experiments can vary in subtle details, the three concepts allow us to organize existing work into congruent categories.…”

Section: Three Core Concepts For Measuring Human Understandingmentioning

confidence: 99%

“…We consider two conditions: 1) whether a person has the perfect knowledge of the task and 2) whether machine predicted labels are revealed. Going through these conditions yields a decision tree that describes different scenarios for human-AI decision-making [32].…”

Section: Introductionmentioning

confidence: 99%

Machine Explanations and Human Understanding

Chen¹,

Shi²,

Sharma³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Explanations are hypothesized to improve human understanding of machine learning models and achieve a variety of desirable outcomes, ranging from model debugging to enhancing human decision making. However, empirical studies have found mixed and even negative results. An open question, therefore, is under what conditions explanations can improve human understanding and in what way. Using adapted causal diagrams, we provide a formal characterization of the interplay between machine explanations and human understanding, and show how human intuitions play a central role in enabling human understanding. Specifically, we identify three core concepts of interest that cover all existing quantitative measures of understanding in the context of human-AI decision making: task decision boundary, model decision boundary, and model error. Our key result is that without assumptions about task-specific intuitions, explanations may potentially improve human understanding of model decision boundary, but they cannot improve human understanding of task decision boundary or model error. To achieve complementary human-AI performance, we articulate possible ways on how explanations need to work with human intuitions. For instance, human intuitions about the relevance of features (e.g., education is more important than age in predicting a person's income) can be critical in detecting model error. We validate the importance of human intuitions in shaping the outcome of machine explanations with empirical human-subject studies. Overall, our work provides a general framework along with actionable implications for future algorithmic development and empirical experiments of machine explanations. CCS Concepts: • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence; Cognitive science; Theory of mind; • Human-centered computing → HCI theory, concepts and models.

show abstract

“…We base our proposal on currently popular AI systems (e.g., decision support, task assistance, recommender systems) and common system features in production and literature. A recent survey paper [32] maps out AI system elements that have been empirically studied for AI decision support in HCI and AI literature, including different types of prediction output, information about the prediction (e.g., local explanations, uncertainty information), information about the model (e.g., performance metrics, documentation, model-wide explanations, training data), and user control features (e.g., customization, feedback to improve the model). Accordingly, we suggest three types of common affordances of AI systems: AI-generated content, transparency, and interaction.…”

Section: Affordances For Trustworthiness Cues: How Is Trustworthiness...mentioning

confidence: 99%

Designing for Responsible Trust in AI Systems: A Communication Perspective

Liao,

Sundar

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current literature and public discourse on "trust in AI" are often focused on the principles underlying trustworthy AI, with insufficient attention paid to how people develop trust. Given that AI systems differ in their level of trustworthiness, two open questions come to the fore: how should AI trustworthiness be responsibly communicated to ensure appropriate and equitable trust judgments by different users, and how can we protect users from deceptive attempts to earn their trust? We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH, which describes how trustworthiness is communicated in AI systems through trustworthiness cues and how those cues are processed by people to make trust judgments. Besides AI-generated content, we highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. By bringing to light the variety of users' cognitive processes to make trust judgments and their potential limitations, we urge technology creators to make conscious decisions in choosing reliable trustworthiness cues for target users and, as an industry, to regulate this space and prevent malicious use. Towards these goals, we define the concepts of warranted trustworthiness cues and expensive trustworthiness cues, and propose a checklist of requirements to help technology creators identify appropriate cues to use. We present a hypothetical use case to illustrate how practitioners can use MATCH to design AI systems responsibly, and discuss future directions for research and industry efforts aimed at promoting responsible trust in AI.

show abstract

Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies

Cited by 43 publications

References 73 publications

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Machine Explanations and Human Understanding

Designing for Responsible Trust in AI Systems: A Communication Perspective

Contact Info

Product

Resources

About