Understanding algorithm aversion: When is advice from automation discounted?

Abstract

Forecasting advice from human advisors is often utilized more than advice from automation. There is little understanding of why “algorithm aversion” occurs, or specific conditions that may exaggerate it. This paper first reviews literature from two fields—interpersonal advice and human–automation trust—that can inform our understanding of the underlying causes of the phenomenon. Then, an experiment is conducted to search for these underlying causes. We do not replicate the finding that human advice is generally utilized more than automated advice. However, after receiving bad advice, utilization of automated advice decreased significantly more than advice from humans. We also find that decision makers describe themselves as having much more in common with human than automated advisors despite there being no interpersonal relationship in our study. Results are discussed in relation to other findings from the forecasting and human–automation trust fields and provide a new perspective on what causes and exaggerates algorithm aversion.

A systematic review of algorithm aversion in augmented decision making

“…Reconciling this body of research is surely important, yet existing work only provides speculative explanations. For instance, Prahl and Van Swol () suggest that their experiment included performance feedback and consistent message characteristics (i.e., only the source description was varied), whereas the set up in Önkal et al () did not include performance feedback and manipulated message characteristics across source types. Understanding how these factors interact with economic incentivization is an empirical question that deserves attention.…”

“…What manifests from these pre-existing expectations is a paradigm in which human decision makers perceive and respond to advice generated by algorithms differently than advice generated by humans, even if the advice itself is identical. Various mechanisms underlying this difference in response are demonstrated throughout the literature, such as the tendency for humans to seek a social or parasocial relationship with the source of advice (Alexander, Blinder, & Zak, 2018;Önkal, Goodwin, Thomson, Gonul, & Pollock, 2009;Prahl & Van Swol, 2017), the persistent belief that human error is random and repairable whereas algorithmic error is systematic (Dietvorst et al, 2015;Dietvorst, Simmons, & Massey, 2016;Highhouse, 2008b), experts' domain confidence leading to underutilization of seemingly unnecessary algorithmic aids (Arkes, Dawes, & Christensen, 1986;Ashton, Ashton, & Davis, 1994), or a lack of training preventing a human user from properly utilizing an algorithmic aid (Mackay & Elam, 1992; cf. Green & Hughes, 1986).…”

“…Essentially, this entails an augmented decision making system in which the human user semisupervises the algorithm by having opportunities to intervene, provide input, and have the final say. As described in the reviewed literature, such decision making systems can take shape in a variety of ways, such as interactive support systems (Lim & O'Connor, 1996), humanautomation systems (Patterson, 2017), engaged systems (Pagano et al, 2016), constructive dialog in expert systems (Eining, Jones, & Loebbecke, 1997), judgmental systems (Prahl & Van Swol, 2017), or procedural presentation in interfaces (Lamberti & Wallace, 1990). Nevertheless, Dietvorst et al (2016) highlight an important new consideration: that people are relatively insensitive to the amount by which they can modify the imperfect algorithm's forecasts as long as they are able to incorporate their own input and participate in the ultimate decision (p. 1161).…”

“…As such, the backfire effect of economic incentives for decision performance, such as that used by Arkes et al (1986), can in fact be considered to be the outcome of putatively rational behavior. However, conflicting results exist in which competitive cash rewards did not lead to algorithm aversion (e.g., Prahl & Van Swol, 2017), as well as experiments where algorithm aversion persists in the absence of competitive economic incentivization (e.g., Önkal et al, 2009) These contrasts lead us to the second source of extrinsic motivation for utilizing algorithmic aids: social incentives. It is widely accepted that decision making is inextricable from the social setting in which it takes place.…”

When Will Workers Follow an Algorithm?: A Field Experiment with a Retail Business

“…Dietvorst et al (2016) found that the algorithm aversion could be eased by ceding a small amount of control. Prahl and Van Swol (2017) theorise the phenomenon in the psyhocological framework. Fehr et al (2013), Bartling et al (2014), and Owens et al (2014) theorize the value of authority and control in the principal-agent framework.…”