Reflecting on Diagnostic Errors: Taking a Second Look is Not Enough

Abstract

BACKGROUND: An experimenter controlled form of reflection has been shown to improve the detection and correction of diagnostic errors in some situations; however, the benefits of participant-controlled reflection have not been assessed. OBJECTIVE: The goal of the current study is to examine how experience and a self-directed decision to reflect affect the accuracy of revised diagnoses. DESIGN: Medical residents diagnosed 16 medical cases (pass 1). Participants were then given the opportunity to reflect on each case and revise their diagnoses (pass 2). PARTICIPANTS: Forty-seven medical Residents in postgraduate year (PGY) 1, 2 and 3 were recruited from Hamilton Health Care Centres. MAIN MEASURES: Diagnoses were scored as 0 (incorrect), 1 (partially correct) and 2 (correct). Accuracies and response times in pass 1 were analyzed using an ANOVA with three factors-PGY, Decision to revise yes/no, and Case 1-16, averaged across residents. The extent to which additional reflection affected accuracy was examined by analyzing only those cases that were revised, using a repeated measures ANOVA, with pass 1 or 2 as a within subject factor, and PGY and Case or Resident as a between-subject factor. KEY RESULTS: The mean score at pass 1 for each level was PGY1, 1.17 (SE 0.50); PGY2, 1.35 (SE 0.67) and PGY3, 1.27 (SE 0.94). While there was a trend for increased accuracy with level, this did not achieve significance. The number of residents at each level who revised at least one diagnosis was 12/19 PGY1 (63 %), 9/11 PGY2 (82 %) and 8/17 PGY3 (47 %). Only 8 % of diagnoses were revised resulting in a small but significant increase in scores from Pass 1 to 2, from 1.20/2 to 1.22 /2 (t=2.15, p=0.03). CONCLUSIONS: Participants did engage in self-directed reflection for incorrect diagnoses; however, this strategy provided minimal benefits compared to knowing the correct answer. Education strategies should be directed at improving formal and experiential knowledge.

To err is human: A case-based review of cognitive bias and its role in clinical decision making

“…16 Several studies have tried to encourage reflective practice and System 2 processes by instructing participants to proceed slowly through their reasoning 18 or by giving participants the opportunity to review their diagnoses. 19 These studies have found minimal or no impact on reducing the rate of diagnostic error. On the other hand, some studies have shown improved diagnostic accuracy when physicians are asked to explicitly state their differential diagnoses along with features that are consistent or inconsistent with each diagnosis.…”

Reflective Practice and Stress: Helpful, Harmful or Uninfluential in Critical Thinking

“…Research demonstrates that some stress and thoughtful reflection are beneficial to the thinking process, 2-4 yet there is no consensus on how to positively manage stress or use reflection in daily clinical care. Two articles 5,6 in this issue of JGIM explore the areas of stress and reflection on critical thinking among trainees.The study by Pottier and colleagues 5 explored the role of stressors on a medical student's history taking, physical examination, and clinical reasoning. The authors performed a study of medical students who participated in two scenarios with standardized patients with familiar disease processes.…”

“…Research demonstrates that some stress and thoughtful reflection are beneficial to the thinking process, 2-4 yet there is no consensus on how to positively manage stress or use reflection in daily clinical care. Two articles 5,6 in this issue of JGIM explore the areas of stress and reflection on critical thinking among trainees.…”

“…In their work on reflection in decision making, Monteiro and colleagues 6 aimed to examine how a resident's clinical experience and self-directed decisions to reflect on their diagnosis affected the accuracy of their decisions. Using a randomized mixed design, 47 medical interns and residents were presented with 16 computerized cases where they were given a patient's primary complaint, medical history, a representative photograph, diagnostic tests and images.…”

Scholarship in Medical Education: Examining our Purpose and Progress

“…The study by Montiero et al reported in this issue suggests that instructing resident physicians to reflect on the most likely diagnosis and giving them more time in the analytical mode does not increase the likelihood that the correct most likely diagnosis will be reached. 5 Classifying medical education research according to its purposes illuminates three levels: the individual study level, the body of studies focusing on the topic, and the entire enterprise of medical education research. Studies that have made the greatest contributions to our knowledge are characterized by a cycle of observation, formulation of a model or hypothesis to explain the results, and predictions of what will happen that are based on the model.…”