Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial

Abstract

Objectives: Cognitive forcing strategies (CFS)may reduce error arising from cognitive biases. This is the first experimental test to determine the effect of CFS training in medical students. Methods: Students were allocated to CFS training or control during a 4-week emergency medicine rotation (n = 191). At the end of the rotation examination, students were tested using computer-based cases. Application of CFS could enable reduction of diagnostic error, as evidenced by identifying multiple correct diagnoses for the two cases prone to search satisficing bias (SSB) and uncommon diagnoses for the two cases prone to availability bias (AB). Two “false positive” cases were included to test for possible “oversearching.” Results: There were 145 students in the intervention and 46 in the control group. For the SSB cases, 52% of students with CFS training and 48% in the control group initiated a search for the second diagnosis (χ2 = 0.13, df = 1, p = 0.91). More than half (54%) correctly identified the second diagnosis in the CFS group, and 48% identified it in the control group. The difference was not significant (χ2 = 2.25, df = 1, p = 0.13). For the second diagnosis in the false positive cases, 64% of the CFS group and 77% of the control group incorrectly identified it. There were no significant differences between groups (χ2 = 2.38, df = 1, p = 0.12). In the AB cases, only 45% in each group identified the uncommon correct diagnosis (χ2 = 0.001, df = 1, p = 0.98). Conclusions: The educational interventions suggested by experts in clinical reasoning and employed in our study to teach CFS failed to show any reduction in diagnostic error by novices.

Reflecting on Diagnostic Errors: Taking a Second Look is Not Enough

“…9,10 However, interventions directed at reducing errors by making clinicians aware of cognitive biases have been negative. 14,15 It may well be that there is no Bquick fix^to reduce errors, and strategies should be directed at improving formal and experiential knowledge.…”

“…2,3,[8][9][10] However, the available scientific evidence does not support this strategy. [14][15][16][17][18][19] Critically, a series of experiments demonstrated that instructing residents to proceed slowly and cautiously when diagnosing written medical cases did not improve diagnostic accuracy compared to being instructed to be more rapid. [16][17][18][19] In another study, participants had improved diagnostic accuracy after receiving a warning about the extreme difficulty of written medical cases compared to participants who did not receive the same warning; however, the authors were not able to replicate this effect with new cases.…”

The evaluation of a continuing professional development package for primary care dentists designed to reduce stress, build resilience and improve clinical decision-making

“…However, there is very little empirical evidence that this is effective and it is therefore controversial. 26,27 However, it might be argued that this is 'the best model we have' at present. A variety of educational techniques to improve clinical reasoning have been researched in general medicine and surgery with mixed results.…”

Understanding Decision Making in Critical Care

“…Indeed, for both control and interventional arms of one CFS study, diagnostic accuracy was inversely related to the time spent on the diagnosis. 58 In contrast, a series of studies conducted by Mamede have shown that structured reflectionsystematically listing the supportive, contradictory, and absent findings of an initial and alternate diagnoses for a written case-improved both learning and categorization of illness scripts in medical students. 59 The reflective intervention was similarly tested in internal medicine residents where it has some success in overcoming availability bias and improving diagnostic accuracy.…”

“…Both uncontrolled and controlled studies have failed to demonstrate clinically or mathematically significant degrees of improvement following this type of intervention. [56][57][58] One interpretation of the results is that the experimental subjects (all students) may not necessarily use non-analytic processing as a primary decision strategy, and if using System II, a "ceiling effect" may exist where further analysis may fail to improve accuracy. Indeed, for both control and interventional arms of one CFS study, diagnostic accuracy was inversely related to the time spent on the diagnosis.…”