Bayes’ theorem: A negative example of a RCT on grommets in children with glue ear

Abstract

Bayesian inference presupposes that practitioners' belief in the effectiveness of medical intervention is the product of prior belief and recent evidence from studies. Although increasingly used, up to now the posterior belief calculated according to the theorem has not been compared with an empirically measured posterior belief. We conducted a RCT, which was preceded by elicitation of prior beliefs among ENT-surgeons, and which was followed by elicitation of posterior beliefs among ENT-surgeons, 1 year after completion of the trial. We compared the posterior beliefs of ENT-surgeons about the effect of grommets in children with glue ears, as predicted by Bayes' theorem with actual measured posterior beliefs. The distribution of the measured posterior beliefs was not in line with the calculated posterior, but almost identical to the distribution of the measured prior beliefs. The results showed that our trial had little or no impact on the beliefs of the ENT-surgeons, i.e. they did not adjust their belief to the extent that was expected according to Bayes' theorem.

Interventions in health care professionals to improve treatment in children with upper respiratory tract infections

“…This results suggest that any active intervention to implement evidence is more effective than passive dissemination of evidence based interventions [10,11]. Other systematic reviews on implementation strategies in general, however, report on ineffective strategies or strategies with an uncertain or variable effectiveness [24,25].…”

“…Passive dissemination of trial results and meta-analyses, may therefore not reach these professionals [9], nor affect how they manage their patients [10,11] In general, clinical practice guidelines are issued to this matter. Alternative strategies comprise educational materials, education sessions, audits and feedbacks, reminders and computerized decision support systems.…”

Methods to elicit beliefs for Bayesian priors: a systematic review

“…Participants had been asked to estimate the probability of an outcome/ event [14,15,19,24,28,42e44,46], the proportion of individuals who will have an outcome [3,17,47], the relative risk of an outcome [35,36], the value for a dependent variable given specified values for independent variables [22,23], and their weight of belief [5,38,39,41]. Commonly used response options include direct probability estimates [4,15], visual analog scale [18,29,42,43], sketching of a graph [16,20], and use of ''bins and chips'' (participants are asked to put the weight of their belief expressed as percentages into discrete intervals [ Fig. 2]) [25,38,41,48].…”

“…Conservatism may occur when clinicians' beliefs confer less certainty to their belief than is justified by the data [43] Include representation of the spectrum of belief…”

“…Provide an example [4,24,39] or training exercise [27,43] Ease of use, clarity Use clear instructions [31] and/or standardized script [44] Anchoring bias: the reported belief is influenced by presentation of data/ scenario [37] Avoid scenarios or summary of data Ordering: participants' probability estimates are influenced by data presented at the beginning of the question stem ( primacy effect) while others are influenced by data presented at the end of the question stem (recency effect) [37] Avoid scenarios or summary of data or scramble the sequence of data presentation between participants [20] Choice of response option Normative goodness Provision of feedback, verification, opportunity for revision [17,20,53] Base-rate neglect: occurs when participants fail to take account of the prevalence of the outcome among untreated patients [15,21] State baseline rate or outcome in untreated patients [15] Summarizing the data Overoptimism, overconfidence Use averaging methods for the group clinical prior [52] …”