Improving medical residents’ self-assessment of their diagnostic accuracy: does feedback help?

Abstract

When physicians do not estimate their diagnostic accuracy correctly, i.e. show inaccurate diagnostic calibration, diagnostic errors or overtesting can occur. A previous study showed that physicians’ diagnostic calibration for easy cases improved, after they received feedback on their previous diagnoses. We investigated whether diagnostic calibration would also improve from this feedback when cases were more difficult. Sixty-nine general-practice residents were randomly assigned to one of two conditions. In the feedback condition, they diagnosed a case, rated their confidence in their diagnosis, their invested mental effort, and case complexity, and then were shown the correct diagnosis (feedback). This was repeated for 12 cases. Participants in the control condition did the same without receiving feedback. We analysed calibration in terms of (1) absolute accuracy (absolute difference between diagnostic accuracy and confidence), and (2) bias (confidence minus diagnostic calibration). There was no difference between the conditions in the measurements of calibration (absolute accuracy, p = .204; bias, p = .176). Post-hoc analyses showed that on correctly diagnosed cases (on which participants are either accurate or underconfident), calibration in the feedback condition was less accurate than in the control condition, p = .013. This study shows that feedback on diagnostic performance did not improve physicians’ calibration for more difficult cases. One explanation could be that participants were confronted with their mistakes and thereafter lowered their confidence ratings even if cases were diagnosed correctly. This shows how difficult it is to improve diagnostic calibration, which is important to prevent diagnostic errors or maltreatment.

Ear, Nose and Throat (ENT) disease diagnostic error in low-resource health care: Observations from a hospital-based cross-sectional study

“…In another study, more than half of paediatricians reported diagnostic error at least once or twice a month, and they made harmful errors at least once or twice a year [ 28 ]. Further, a study in the Netherlands reported diagnostic accuracy of 0.42 among medical residents, scored as either 0 (incorrect), 0.5 (partially correct) or 1 (correct) [ 29 ]. With 96.7% of allergic rhinitis, 95.7% of laryngopharyngeal reflux and 87.0% of ear wax impaction missed by non-ENT clinicians in our study, most of the patients with ENT conditions are likely inappropriately treated with the wrong medication and surgery.…”

Gender Differences in Perceived Working Conditions of General Practitioners During the COVID-19 Pandemic—a Cross-Sectional Study

“…Kuhn et al found that feedback made physicians more uncertain of themselves, resulting in an increase in underconfidence, while Nederhand et al observed the opposite effect. 43 , 44 …”

“…42 As a result, further unnecessary examinations and tests are carried out, lengthening the diagnostic process. 43 The extent to which additional training helps mitigate underconfidence is unclear. Kuhn et al found that feedback made physicians more uncertain of themselves, resulting in an increase in underconfidence, while Nederhand et al observed the opposite effect.…”

“…Kuhn et al found that feedback made physicians more uncertain of themselves, resulting in an increase in underconfidence, while Nederhand et al observed the opposite effect. 43,44 As with confidence, how GPs feel about taking risks may have a significant impact on how clinical decisions are made. Attitudes towards risk and the perception of risk were found to influence hospital admission rates, the use of laboratory tests, the use of imaging in emergency departments, and the willingness to prescribe medications and update immunizations.…”

Impact of performance and information feedback on medical interns' confidence–accuracy calibration

“…Performance feedback in clinical practice can be as simple as determining if the patient’s treatment was successful or not; information feedback expands on that and could include additional information such as what treatment was ultimately successful or which follow-up tests or results provided further insights. Despite the fact that research specifically concerning calibration and feedback on the diagnostic process remains scarce, previous studies have shown that performance feedback could improve calibration on easy clinical cases (Nederhand et al, 2018 ) but not on difficult cases (Kuhn et al, 2022 ). It has been suggested that information feedback is needed to improve the diagnostic process (Archer, 2010 ; Ryan et al, 2020 ), though evidence for its effects remains limited (Kornegay et al, 2017 ).…”