Understanding Algorithm Aversion: Forecasters Erroneously Avoid Algorithms After Seeing them Err

Dietvorst, Berkeley J.; Simmons, Joseph P.; Massey, Cade

doi:10.5465/ambpp.2014.12227abstract

Cited by 23 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Are these fears warranted? Even if machines were to give unethical advice, a phenomenon BAD MACHINES CORRUPT GOOD MORALS which has yet to be documented, we know that people state that they are not necessarily keen on following algorithmic recommendations in non-technical domains 70,71 . While this aversion could, in theory, dampen the effect of unethical machine advice, recent evidence from a large-scale experiment tells a different story 72 .…”

Section: Advisormentioning

confidence: 99%

Bad machines corrupt good morals

Köbis

Bonnefon²,

Rahwan³

2021

Nat Hum Behav

View full text Add to dashboard Cite

Machines powered by Artificial Intelligence (AI) are now influencing the behavior of humans in ways that are both like and unlike the ways humans influence each other. In light of recent research showing that other humans can exert a strong corrupting influence on people's ethical behavior, worry emerges about the corrupting power of AI agents. To estimate the empirical validity of these fears, we review the available evidence from behavioral science, human-computer interaction, and AI research. We propose that the main social roles through which both humans and machines can influence ethical behavior are (a) role model, (b) advisor, (c) partner, and (d) delegate. When AI agents become influencers (role models or advisors), their corrupting power may not exceed (yet) the corrupting power of humans. However, AI agents acting as enablers of unethical behavior (partners or delegates) have many characteristics that may let people reap unethical benefits while feeling good about themselves, indicating good reasons for worry. Based on these insights, we outline a research agenda that aims at providing more behavioral insights for better AI oversight.

show abstract

Section: Advisormentioning

confidence: 99%

Bad machines corrupt good morals

Köbis

Bonnefon²,

Rahwan³

2021

Nat Hum Behav

View full text Add to dashboard Cite

show abstract

“…61 Conversely, algorithm aversion occurs when humans disregard algorithms that actually perform better than humans, thus affecting trust calibration in the opposite direction to automation bias. 62 This effect has been studied most in the context of forecasting tasks, whereby humans tend to lose trust in an algorithm's advice very rapidly in response to errors; 63 by contrast, trust in other humans who make the same errors reduces more slowly. 64 Other experiments have produced conflicting results, suggesting that only expert forecasters are susceptible to algorithm aversion while lay users are more likely to trust algorithmic advice.…”

Section: Main Textmentioning

confidence: 99%

Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI

Tomsett¹,

Preece

Braines³

et al. 2020

Patterns

View full text Add to dashboard Cite

Artificial intelligence (AI) systems hold great promise as decision-support tools, but we must be able to identify and understand their inevitable mistakes if they are to fulfill this potential. This is particularly true in domains where the decisions are high-stakes, such as law, medicine, and the military. In this Perspective, we describe the particular challenges for AI decision support posed in military coalition operations. These include having to deal with limited, low-quality data, which inevitably compromises AI performance. We suggest that these problems can be mitigated by taking steps that allow rapid trust calibration so that decision makers understand the AI system's limitations and likely failures and can calibrate their trust in its outputs appropriately. We propose that AI services can achieve this by being both interpretable and uncertainty-aware. Creating such AI systems poses various technical and human factors challenges. We review these challenges and recommend directions for future research.

show abstract

“…Understanding how we react to mistakes by machines (as compared to those by humans) is not an easy task. There is strong evidence that people react differently to mistakes made by machines and humans ( Dietvorst et al., 2014 , 2015 ; Malle et al, 2015 ; Awad et al., 2020 ). There are also reasons to believe that people assign blame differently based on the difficulty of encountered situations.…”

Section: Introductionmentioning

confidence: 99%

Blaming automated vehicles in difficult situations

2021

View full text Add to dashboard Cite

Summary Automated vehicles (AVs) have made huge strides toward large-scale deployment. Despite this progress, AVs continue to make mistakes, some resulting in death. Although some mistakes are avoidable, others are hard to avoid even by highly skilled drivers. As these mistakes continue to shape attitudes toward AVs, we need to understand whether people differentiate between them. We ask the following two questions. When an AV makes a mistake, does the perceived difficulty or novelty of the situation predict blame attributed to it? How does that blame attribution compare to a human driving a car? Through two studies, we find that the amount of blame people attribute to AVs and human drivers is sensitive to situation difficulty. However, while some situations could be more difficult for AVs and others for human drivers, people blamed AVs more, regardless. Our results provide novel insights in understanding psychological barriers influencing the public's view of AVs.

show abstract

Understanding Algorithm Aversion: Forecasters Erroneously Avoid Algorithms After Seeing them Err

Cited by 23 publications

References 0 publications

Bad machines corrupt good morals

Bad machines corrupt good morals

Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI

Blaming automated vehicles in difficult situations

Contact Info

Product

Resources

About