Why verifying diagnostic decisions with a checklist can help: insights from eye tracking

Abstract

Making a diagnosis involves ratifying or verifying a proposed answer. Formalizing this verification process with checklists, which highlight key variables involved in the diagnostic decision, is often advocated. However, the mechanisms by which a checklist might allow clinicians to improve their verification process have not been well studied. We hypothesize that using a checklist to verify diagnostic decisions enhances analytic scrutiny of key variables, thereby improving clinicians’ ability to find and fix mistakes. We asked 16 participants to verify their interpretation of 12 electrocardiograms, randomly assigning half to be verified with a checklist and half with an analytic prompt. While participants were verifying their interpretation, we tracked their eye movements. We analyzed these eye movements using a series of eye tracking variables theoretically linked to analytic scrutiny of key variables. We found that more errors were corrected using a checklist compared to an analytic prompt (.27 ± .53 errors per ECG vs. .04 ± .43, F1,15 = 8.1, p = .01, η2 = .20). Checklist use was associated with enhanced analytic scrutiny in all eye tracking measures assessed (F6,10 = 6.0, p = .02). In this experiment, using a key variable checklist to verify diagnostic decisions improved error detection. This benefit was associated with enhanced analytic scrutiny of those key variables as measured by eye tracking.

A review of eye tracking for understanding and improving diagnostic interpretation

“…Over the past four decades since this original research, eye tracking has been expanded to understanding diagnostic interpretation in several medical specializations, including radiology, breast pathology, general surgery, neurology, emergency medicine, anesthesiology, ophthalmology, and cardiology (Balslev et al, 2012; Berbaum et al, 2001; Brunyé et al, 2014; Giovinco et al, 2015; Henneman et al, 2008; Jungk, Thull, Hoeft, & Rau, 2000; Krupinski et al, 2006; Kundel, Nodine, Krupinski, & Mello-Thoms, 2008; Matsumoto et al., 2011; O’Neill et al, 2011; Sibbald, de Bruin, Yu, & van Merrienboer, 2015; Wood, Batt, Appelboam, Harris, & Wilson, 2014). In general, these eye-tracking studies have found evidence of reliable distinctions between three types of error-making in diagnostic interpretation: search errors, recognition errors, and decision errors.…”

To err is human: A case-based review of cognitive bias and its role in clinical decision making

“…One randomized controlled trial failed to show a statistically significant reduction in the diagnostic error rate with the use of checklists, except in a small subgroup of patients treated in the ED. 21 These findings challenge the results of two other studies that found checklists to be effective in improving scrutiny 22 and diagnostic accuracy 23 when interpreting electrocardiograms. More advanced forms of clinician decision support systems have also been studied.…”

Diagnostic Reboot: A Proposal to Improve Diagnostic Reasoning

“…Previously, Sibbald et al have shown that checklists helped to find and fix mistakes in physicians of all levels, and physicians with minimal experience were shown to benefit more [ 10 ]. The mechanism for this is likely related to enhanced analytic scrutiny and a more careful examination of minor details [ 11 ]. Our findings support previous findings by Myung et al that analytic reasoning reduced cognitive errors at least among novice physicians.…”