Physician Bayesian updating from personal beliefs about the base rate and likelihood ratio

Abstract

Whether humans can accurately make decisions in line with Bayes' rule has been one of the most important yet contentious topics in cognitive psychology. Though a number of paradigms have been used for studying Bayesian updating, rarely have subjects been allowed to use their own preexisting beliefs about the prior and the likelihood. A study is reported in which physicians judged the posttest probability of a diagnosis for a patient vignette after receiving a test result, and the physicians' posttest judgments were compared to the normative posttest calculated from their own beliefs in the sensitivity and false positive rate of the test (likelihood ratio) and prior probability of the diagnosis. On the one hand, the posttest judgments were strongly related to the physicians' beliefs about both the prior probability as well as the likelihood ratio, and the priors were used considerably more strongly than in previous research. On the other hand, both the prior and the likelihoods were still not used quite as much as they should have been, and there was evidence of other nonnormative aspects to the updating, such as updating independent of the likelihood beliefs. By focusing on how physicians use their own prior beliefs for Bayesian updating, this study provides insight into how well experts perform probabilistic inference in settings in which they rely upon their own prior beliefs rather than experimenter-provided cues. It suggests that there is reason to be optimistic about experts' abilities, but that there is still considerable need for improvement.

Physician and Nonphysician Estimates of Positive Predictive Value in Diagnostic v. Mass Screening Mammography: An Examination of Bayesian Reasoning

“…GPs generally reasoned that the probability of disease is higher given a symptom and a positive test than when given only a positive test or only a symptom. Contrary to previous studies that find physicians ignore base rates when estimating PPV, this study, like Rottman's, 16 finds that they consider prior information about disease status when estimating probability of disease following a positive test. The question, then, is what accounts for the difference in findings.…”

“…Breast cancer screening was generally not personally very relevant to the MBAs; a similar study with women of mass screening age would address this concern. An alternative explanation for overestimation of PPV S is that pretest probabilities, especially in screening, were overestimated and that intuitive Bayesian analysis led to less perceived than actual difference; Rottman 16 offers an approach to examine this. The survey design may have led to overestimation of pretest probabilities because of hindsight bias 33 ; it would be better to ask pretest probability before revealing a test result.…”

“…However, while a posttest probability of . 16 following a positive test reflects a great increase over a pretest probability of .005, not having the condition is still 6 times more likely than having it.) Representative thinking leads people to ignore or insufficiently account for prior beliefs about population membership when updating those beliefs given new, individuating information (e.g., a test result).…”

“…Motivated by similar concerns, Rottman 16 first elicited physicians' pretest and posttest probability estimates and then elicited estimates of sensitivity and the false-positive rate. He found that posttest judgments were predicted by judged pretest probability and elicited the likelihood ratio (sensitivity divided by false-positive rate).…”

Análisis bayesiano. Conceptos básicos y prácticos para su interpretación y uso

“…Mediante los ejemplos anteriores hemos mostrado algunas ventajas de la estadística bayesiana sobre la frecuentista. La primera es que constituye un método más intuitivo para comprender cómo una investigación no modifi ca, aumenta o disminuye los conocimientos sobre un problema científi co. 8,29,30,31 La segunda es que permite incluir la información previa a un estudio en la evaluación y conclusiones del mismo, situación no tomada en cuenta en el análisis clásico o frecuentista. 2,7 Tercero, el método frecuentista se basa en el supuesto de repetir los estudios varias veces, situación que nunca sucede.…”