Cognitive Heuristics and Feedback in a Dynamic Decision Environment

Abstract

Research on cognitive processes in decision making has identified heuristics that often work well but sometimes lead to serious errors. This paper presents an investigation of the performance of heuristics in a complex dynamic setting, characterized by repeated decisions with feedback. There are three components: (1) A simulated task resembling medical decision problems (diagnosis and treatment) is described. (2) Computer models of decision strategies are developed. These include models based on cognitive heuristics as well as benchmark strategies that indicate the limit of the heuristic strategies' performance. The upper benchmark is based on statistical decision theory, the lower one on random trial and error. (3) Selected task characteristics are systematically varied and their influence on performance evaluated in simulation experiments. Results indicate that task characteristics often studied in past research (e.g., symptom diagonosticity, disease base-rates) have less influence on performance relative to feedback-related aspects of the task. These dynamic characteristics are a major determinant of when heuristics perform well or badly. The results also provide insights about the costs and benefits of various cognitive heuristics. In addition, the possible contribution of this research to the design and evaluation of decision aids is considered.decision strategies, heuristics, medical decisions

Why don’t well-educated adults understand accumulation? A challenge to researchers, educators, and citizens

“…Furthermore, learning in dynamic systems is often slow and weak, even with repeated trials, unlimited time, and performance incentives (Diehl & Sterman, 1995;Kleinmuntz & Schkade, 1993;Sterman, 1989aSterman, , 1989b. Many of these studies involved tasks of great complexity, and poor performance was often ascribed to the large number of entities and interactions, feedback delays, and information overload (Brehmer, 1990(Brehmer, , 1995Gonzalez, 2005a;Kleinmuntz, 1985;Omodei & Wearing, 1995).…”

“…Furthermore, learning in dynamic systems is often slow and weak, even with repeated trials, unlimited time, and performance incentives (Diehl & Sterman, 1995;Kleinmuntz & Schkade, 1993;Sterman, 1989aSterman, , 1989b. Many of these studies involved tasks of great complexity, and poor performance was often ascribed to the large number of entities and interactions, feedback delays, and information overload (Brehmer, 1990(Brehmer, , 1995Gonzalez, 2005a;Kleinmuntz, 1985;Omodei & Wearing, 1995).More recent work, however, has shown that people make persistent mistakes even in the simplest dynamic systems, including systems consisting of one stock, one inflow, and one outflow, with no feedback processes, time delays, or nonlinearities (e.g., Booth Sweeney & Sterman, 2000;Cronin & Gonzalez, 2007;Sterman & Booth Sweeney, 2007). To illustrate, the ''department store" task (Sterman, 2002) presents participants with a graph showing the number of people entering and leaving a department store each minute over a 30-min interval (Fig.…”

Misperceptions of Feedback in Dynamic Decision Making

“…Experiments in both economics and psychology have focussed (with significant exceptions, e.g. Hogarth and Makridakis 1981, Kleinmuntz 1985, Brehmer 1986, Smith 1986) on static and discrete judgments.…”

“…Experiments in both economics and psychology have focussed (with significant exceptions, e.g. Hogarth and Makridakis 1981, Kleinmuntz 1985, Brehmer 1986, Smith 1986) on static and discrete judgments.Hogarth (1981, 198) emphasizes ...the continuous, adaptive nature of the judgmental processes used to cope with a complex, changing environment... With few exceptions...judgment researchers have focussed on discrete incidents (particular actions, predictions, and choices) that punctuate these continuous processes; furthermore, task environments are typically conceptualized to be stable.... [l]nsufficient attention has been paid' to the effect of feedback between organism and environment.The complexity and scale of corporate and economic systems renders experiments on the systems itself infeasible. This paper argues that experimental studies of the "feedback between organism and environment" in aggregate dynamic systems such as the economy can be conducted in the laboratory with computer simulation models.…”

Implementing Welfare-to-Work Services: A Study of Staff Decision-Making

“…Case vignettes have been used to assess decision making across a range of clinical and professional settings, including welfare programs (Hagen 8c Owens-Manley, The decisions made by welfare-to-work staff can be seen as dynamic decisions (Kleinmuntz, 1985) in which a series of related decisions and actions inform each other over a period of time. Carroll and Johnson (1990) Table 1).…”