What is learned from experience in a probabilistic environment?

Abstract

Three experiments explored what is learned from experience in a probabilistic environment. The task was a simulated medical decision-making task with each patient having one of two test results and one of two diseases. The test result was highly predictive of the disease for all participants. The base rate of the test result was varied between participants to produce different inverse conditional probabilities of the test result given the disease across conditions. Participants trained using feedback to predict a patient's disease from a test result showed the classic confusion of the inverse error, substituting the forward conditional probability for the inverse conditional probability when tested on it. Additional training on the base rate of the test result did little to improve performance. Training on the joint probabilities, however, produced good performance on either conditional probability. The pattern of results demonstrated that experience with the environment is not always sufficient for good performance. That natural sampling leads to good performance was not supported. Further, because participants not trained on joint probabilities did, however, know them but still committed the confusion of the inverse error, the hypothesis that having joint probabilities would facilitate performance was not supported. The pattern of results supported the conclusion that people learn all the necessary information from experience in a probabilistic environment, but depending upon what the experience was, it may interfere with their ability to recall to memory the appropriate sample set necessary for estimating or using the inverse conditional probability. Copyright # 2004 John Wiley & Sons, Ltd. key words decision making; direct experience; conditional probabilities; confusion of the inverse Making judgments that lead to decisions impacts on every area of human endeavor. Yet it is well known that humans are very susceptible to a multitude of systematic errors in processing uncertain (or probabilistic) information when making judgments. Much research has shown that these errors are common and easily produced. More importantly, this research has provided insight into the cognitive processes people use in

Physician Bayesian updating from personal beliefs about the base rate and likelihood ratio

“…For example, when considering the probability that a patient has colorectal cancer after receiving a negative FOBT test, a physician could think back to a set of similar patients (who have a similar pretest likelihood or a similar set of demographic characteristics and symptoms) and who received a negative FOBT test, and estimate the percentage of these patients who eventually were found to have colorectal cancer. This process is believed to be the process participants use when they have access to extensive prior Bnatural^experience (Kleiter, 1994) or even lab-based trial-by-trial learning experience (Edgell et al, 2004; see also Barbey & Sloman, 2007;Gigerenzer & Hoffrage, 1995;Kleiter et al, 1997). Noisy recall of these quantities could also contribute to the imperfect coherence between the pretest, likelihood, and posttest judgments (Hilbert, 2012).…”

“…Subsequent work has found that a number of factors can lead to improved Bayesian reasoning (Gigerenzer & Hoffrage, 1995;Krynski & Tenenbaum, 2007). In particular, one setting that can lead to improved posterior probability judgments is when individuals experience the contingency between the two variables in question (e.g., mammogram result and having cancer) in a trial-by-trial learning paradigm (ChristensenSzalanski & Beach, 1982;Edgell, Harbison, Neace, Nahinsky, & Lajoie, 2004).…”

Sequentially simulated outcomes: Kind experience versus nontransparent description.

“…Paradoxically, participants in their experiments never actually experienced data sequentially, that is, “as a series of events.” Instead, they observed totals. That is, Gigerenzer and his colleagues presented data in the form of summarized natural frequencies (see also Edgell, Harbison, Neace, Nahinsky, & Lajoie, 2004, p. 214).…”

Theory about Heuristic Strategies Based on Verbal Protocol Analysis: The Emperor Needs a Shave

“…[11][12][13][14] The broader community of medical decision making researchers has not embraced the topic of heuristics and biases approach with sustained enthusiasm. Although some errors attributable to previously identified heuristic strategies have been demonstrated with medical vignettes 15 as well as simulated medical decision-making tasks, 16 others have failed to replicate those errors. 17 There has been an unsteady stream of demonstrations of inaccuracies of physician judgments of probability about their own patients or hypothetical ones, influenced by recency, 18 by the utility of the outcome, 19 or by anchoring on one's original judgments.…”