Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them

Abstract

Although evidence-based algorithms consistently outperform human forecasters, people often fail to use them after learning that they are imperfect, a phenomenon known as algorithm aversion. In this paper, we present three studies investigating how to reduce algorithm aversion. In incentivized forecasting tasks, participants chose between using their own forecasts or those of an algorithm that was built by experts. Participants were considerably more likely to choose to use an imperfect algorithm when they could modify its forecasts, and they performed better as a result. Notably, the preference for modifiable algorithms held even when participants were severely restricted in the modifications they could make (Studies 1–3). In fact, our results suggest that participants’ preference for modifiable algorithms was indicative of a desire for some control over the forecasting outcome, and not for a desire for greater control over the forecasting outcome, as participants’ preference for modifiable algorithms was relatively insensitive to the magnitude of the modifications they were able to make (Study 2). Additionally, we found that giving participants the freedom to modify an imperfect algorithm made them feel more satisfied with the forecasting process, more likely to believe that the algorithm was superior, and more likely to choose to use an algorithm to make subsequent forecasts (Study 3). This research suggests that one can reduce algorithm aversion by giving people some control—even a slight amount—over an imperfect algorithm’s forecast. Data, as supplemental material, are available at https://doi.org/10.1287/mnsc.2016.2643 . This paper was accepted by Yuval Rottenstreich, judgment and decision making.

A systematic review of algorithm aversion in augmented decision making

“…Regarding algorithmic decision aids, studies such as these highlight the need for affording real or perceived decision control to the human user in order to satisfy his or her psychological needs and self‐interest (Colarelli & Thompson, ). In fact, this conclusion corraborates Dietvorst et al's (, ) finding that trust in an algorithm degrades quickly upon seeing it err, but that it can be equally quick to restore by allowing the human decision maker to modify the algorithm's judgment, even under constraints. Here, algorithm aversion appears to manifest itself in augmented decision‐making systems that fail to address human users' psychological need for agency, autonomy, and control.…”

“…In large part, the recent findings of Dietvorst et al () are a rework of an old concept: human‐in‐the‐loop decision making. Essentially, this entails an augmented decision making system in which the human user semisupervises the algorithm by having opportunities to intervene, provide input, and have the final say.…”

Understanding algorithm aversion: When is advice from automation discounted?

“…Recently, this effect has been noted in forecasting research (Önkal et al, ) and has been called algorithm aversion (Dietvorst, Simmons, & Massey, ). A developing area of research is trying to identify interventions that increase trust in automation advice, such as providing confidence intervals or allowing human judges to slightly modify automation forecasts (Dietvorst, Simmons, & Massey, ; Goodwin, Gönül, & Önkal, ). This research is important, but more research is needed on the underlying psychological processes that affect the discounting of automation advice, especially in comparison to human advice.…”

The Enemy of Good: Estimating the Cost of Waiting for Nearly Perfect Automated Vehicles

“…This may not be acceptable to society. A large body of research suggests that peoples' willingness to accept technological risk is governed by factors related not only to the actual risk but also to other characteristics (Sjöberg, 2000;Slovic and Peters, 2006;Dietvorst, Simmons, and Massey, 2014). For example, risks are more acceptable when they are voluntary (which it may not be for the many road users who will have to share the road with HAVs) and if a person can exert control over the outcomes (which is, by definition, not the case for higher levels of vehicle automation) (Starr, 1969;Fischhoff et al, 1978;Otway and von Winterfeldt, 1982;Slovic, 1987Slovic, , 2000Dietvorst, Simmons, and Massey, 2016).…”