Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel

Abstract

BackgroundDiagnostic errors unfortunately remain common. Electronic differential diagnostic support (EDS) systems may help, but it is unclear when and how they ought to be integrated into the diagnostic process.ObjectiveTo explore how much EDS improves diagnostic accuracy, and whether EDS should be used early or late in the diagnostic process.Setting6 Canadian medical schools. A volunteer sample of 67 medical students, 62 residents in internal medicine or emergency medicine, and 61 practising internists or emergency medicine physicians were recruited in May through June 2020.InterventionParticipants were randomised to make use of EDS either early (after the chief complaint) or late (after the complete history and physical is available) in the diagnostic process while solving each of 16 written cases. For each case, we measured the number of diagnoses proposed in the differential diagnosis and how often the correct diagnosis was present within the differential.ResultsEDS increased the number of diagnostic hypotheses by 2.32 (95% CI 2.10 to 2.49) when used early in the process and 0.89 (95% CI 0.69 to 1.10) when used late in the process (both p<0.001). Both early and late use of EDS increased the likelihood of the correct diagnosis being present in the differential (7% and 8%, respectively, both p<0.001). Whereas early use increased the number of diagnostic hypotheses (most notably for students and residents), late use increased the likelihood of the correct diagnosis being present in the differential regardless of one’s experience level.Conclusions and relevanceEDS increased the number of diagnostic hypotheses and the likelihood of the correct diagnosis appearing in the differential, and these effects persisted irrespective of whether EDS was used early or late in the diagnostic process.

Incidence of Diagnostic Errors Among Unexpectedly Hospitalized Patients Using an Automated Medical History–Taking System With a Differential Diagnosis Generator: Retrospective Observational Study

“…To date, several studies have been conducted to evaluate the impact of the timing of using a diagnostic decision support system. According to their studies, physician diagnosis was associated with their first impression [ 50 ], and early use of diagnostic support systems before collecting information by physicians significantly improved the diagnostic accuracy [ 21 - 23 ]. These findings may support the positive effects of the implementation of an automated medical history–taking system with a differential diagnosis generator, which can provide diagnostic decision support before physicians collect information.…”

“…Computerized automated history-taking systems perform better in clinical documentation tasks for taking patient histories than do physicians [ 19 , 20 ]. The use of a diagnostic support system (ie, differential diagnosis generator) before collecting information by physicians showed a significant impact on the improvement of diagnostic accuracy in terms of clinical reasoning and differential diagnosis [ 21 - 23 ]. Moreover, a new system that combines automated medical history–taking functions with differential diagnosis generation—specialized for musculoskeletal diseases only—showed improved diagnostic accuracy among physicians in a pilot randomized controlled trial [ 24 ].…”

Data Integration Challenges for Machine Learning in Precision Medicine

“…However, many researchers and clinicians have become skeptical of the tool due to initial over-promises from the company ( 164 ). Many other diagnostic support applications have been developed, most of them aimed at commercial use such is the case of ISABEL https://www.isabelhealthcare.com/ ( 165 , 166 ) and others. However, due to commercial restrictions, their AI/ML assessment and their use in LSDA has been rather restricted ( 167 , 168 ).…”

Research to improve diagnosis: time to study the real world

“…improve diagnostic accuracy. 12 However, prior research on the effectiveness of CDS at preventing other types of adverse events has shown that CDS typically have a small impact on the quality and safety of care. This is because CDS are often implemented without accounting for the social and technical context in which they will be used.…”

“…Many hospitals—including ours—already have technology-based system interventions (such as electronic health record (EHR) alerts based on physiological variables) to help clinicians identify patients with sepsis, with the goal of initiating evidence-based treatment in a timely fashion. There is some evidence in simulation studies that the use of clinician decision support systems (CDS) that generate a list of possible diagnoses can help improve diagnostic accuracy 12. However, prior research on the effectiveness of CDS at preventing other types of adverse events has shown that CDS typically have a small impact on the quality and safety of care.…”