The Effectiveness of Electronic Differential Diagnoses (DDX) Generators: A Systematic Review and Meta-Analysis

Abstract

BackgroundDiagnostic errors are costly and they can contribute to adverse patient outcomes, including avoidable deaths. Differential diagnosis (DDX) generators are electronic tools that may facilitate the diagnostic process.Methods and FindingsWe conducted a systematic review and meta-analysis to investigate the efficacy and utility of DDX generators. We undertook a comprehensive search of the literature including 16 databases from inception to May 2015 and specialist patient safety databases. We also searched the reference lists of included studies. Article screening, selection and data extraction were independently conducted by 2 reviewers. 36 articles met the eligibility criteria and the pooled accurate diagnosis retrieval rate of DDX tools was high with high heterogeneity (pooled rate = 0.70, 95% CI = 0.63 to 0.77; I2 = 97%, p<0.0001). DDX generators did not demonstrate improved diagnostic retrieval compared to clinicians but small improvements were seen in the before and after studies where clinicians had the opportunity to revisit their diagnoses following DDX generator consultation. Clinical utility data generally indicated high levels of user satisfaction and significant reductions in time taken to use for newer web-based tools. Lengthy differential lists and their low relevance were areas of concern and have the potential to increase diagnostic uncertainty. Data on the number of investigations ordered and on cost-effectiveness remain inconclusive.ConclusionsDDX generators have the potential to improve diagnostic practice among clinicians. However, the high levels of heterogeneity, the variable quality of the reported data and the minimal benefits observed for complex cases suggest caution. Further research needs to be undertaken in routine clinical settings with greater consideration of enablers and barriers which are likely to impact on DDX use before their use in routine clinical practice can be recommended.

The global burden of diagnostic errors in primary care

“…Interventions that could potentially improve clinical reasoning include advice to practice reflectively, [70][71][72] use of tools such as mnemonics and checklists, 73 and web-based decision support products that assist with differential diagnosis. [74][75][76] 'Debiasing' has been proposed as an approach to eliminate subconscious biases that contribute to error. 77 78 While many find the idea of focusing on cognitive biases and heuristics to improve clinical reasoning appealing, physicians find it difficult to characterise specific biases with any reliability and identify twice as many biases when outcome implies incorrect versus correct diagnosis.…”

Diagnosing Crime and Diagnosing Disease: Bias Reduction Strategies in the Forensic and Clinical Sciences

“…Such lists need careful construction, nonhierarchical team effort and adherence. Checklists have succeeded in reducing infections in intensive care units and operating rooms, but they fail if there is a lack of adherence . Continued reliance on human cognition: Even with its bias and inherent failures, human cognition is usually better than computer‐generated differential diagnostic algorithms and symptom checkers for office‐based clinician–patient encounters . Artificial intelligence is not, as yet, a solution for all errors or biases. Blinding: Visually based interpretive tasks, such as that of a pathologist examining specimens or a radiologist detecting imaging patterns, may benefit from a minimum of essential information from treating clinicians.…”

“…Outcomes from the use of checklists have been mixed: Some research in real-world settings has demonstrated their utility and has provided a useful window on the expert's decision-making process (44,52,53,56,57). Other studies have found checklists to be of limited effectiveness in reducing errors in clinical decisionmaking or procedures (4,(58)(59)(60). Researchers have hypothesized that this lack of efficacy is due to underlying limitations within the checklist approach (61).…”

“…• Continued reliance on human cognition: Even with its bias and inherent failures, human cognition is usually better than computer-generated differential diagnostic algorithms and symptom checkers for office-based clinician-patient encounters (4,101). Artificial intelligence is not, as yet, a solution for all errors or biases.…”

“…Reviews indicate that nonspecific strategies for reducing bias are only minimally effective (3)(4)(5)(6)(7). Therefore, newer focused techniques in decision-making have emerged: These include blinding and the use of specific checklists.…”