User Perceptions of Decision Support Effectiveness: Two Production Planning Experiments*

Abstract

Decision support systems continue to be very popular in business, despite mixed research evidence as to their effectiveness. We hypothesize that what-if analysis, a prominent feature of most decision support systems, creates an "illusion of control" causing users to overestimate its effectiveness. Two experiments involving a production planning task are reported which examine decision makers' perceptions of the effectiveness of what-if analysis relative to the alternatives of unaided decision making, and quantitative decision rules. Experiment 1 found that almost all subjects believed what-if analysis was superior to unaided decision making, although using what-if analysis had no significant effect on performance. Experiment 2 found that decision makers were indifferent between what-if analysis and a quantitative decision rule which, if used, would have led to significant cost savings. Thus, what-if analysis did create an illusion of control: decision makers perceived performance differences where none existed, and did not detect large differences when they were present. In both experiments, decision makers exhibited difficulty realizing that their positive beliefsabout what-if analysis were exaggerated. Such misjudgments could lead people to continue using what-if analysis even when it is not beneficial and to avoid potentially superior decision support technologies.Subject Areas: Decision Support Systems and Production/Operations Management.

The influence of information presentation formats on complex task decision-making performance

“…The two complex tasks consisted of a facility location task (the complex-symbolic task) (Buffa, 1990) and an aggregate planning task (the complex-spatial task) (Holt et al, 1960;Remus, 1984Remus, , 1987Davis and Kotterman, 1994). In the facility location task, subjects were provided with five different cost estimates associated with six warehouse locations.…”

Controlling the illusion of control: a grounded theory of sports betting advertising in the UK

“…In experiments where gamblers have been presented with pseudo-controlling situations (e.g., picking their own number in a lottery; Burger, 1991) or bogus artefacts misconstrued as control-enhancers (e.g., using a reel-stopping device in a video lottery terminal game; Ladouceur & Sévigny, 2005), it has been demonstrated that gamblers consistently prefer these options when presented even if these actions have no instrumental impact on gambling outcomes. In other contexts, researchers have also found that people tend to exaggerate the benefits of devices that enhance the illusion of control, and fail to detect that such devices do not offer any advantage in gambling outcomes (Davis & Kottemann, 1994).…”

Inherent biases in decision support systems: the influence of optimistic and pessimistic DSS on choice, affect, and attitudes

“…The ANOVA of the attitude scale by the experimental variable was not significant. The borderline significance may be explained by Davis and Kottemann's (1994) suggestion that DSS users' attitudes may not relate to its actual effectiveness.…”