Design, implementation, and evaluation of the computer-aided clinical decision support system based on learning-to-rank: collaboration between physicians and machine learning in the differential diagnosis process

Abstract

Background We are researching, developing, and publishing the clinical decision support system based on learning-to-rank. The main objectives are (1) To support for differential diagnoses performed by internists and general practitioners and (2) To prevent diagnostic errors made by physicians. The main features are that “A physician inputs a patient's symptoms, findings, and test results to the system, and the system outputs a ranking list of possible diseases”. Method The software libraries for machine learning and artificial intelligence are TensorFlow and TensorFlow Ranking. The prediction algorithm is Learning-to-Rank with the listwise approach. The ranking metric is normalized discounted cumulative gain (NDCG). The loss functions are Approximate NDCG (A-NDCG). We evaluated the machine learning performance on k-fold cross-validation. We evaluated the differential diagnosis performance with validated cases. Results The machine learning performance of our system was much higher than that of the conventional system. The differential diagnosis performance of our system was much higher than that of the conventional system. We have shown that the clinical decision support system prevents physicians' diagnostic errors due to confirmation bias. Conclusions We have demonstrated that the clinical decision support system is useful for supporting differential diagnoses and preventing diagnostic errors. We propose that differential diagnosis by physicians and learning-to-rank by machine has a high affinity. We found that information retrieval and clinical decision support systems have much in common (Target data, learning-to-rank, etc.). We propose that Clinical Decision Support Systems have the potential to support: (1) recall of rare diseases, (2) differential diagnoses for difficult-to-diagnoses cases, and (3) prevention of diagnostic errors. Our system can potentially evolve into an explainable clinical decision support system.

Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence–Driven Automated History–Taking System: Pilot Cross-Sectional Study

“…The AI diagnostic support system for general internal medicine is a diagnostic generator freely available on the internet. This system uses learning-to-rank prediction algorithms with a listwise approach, which is similar to the DDx process of experienced physicians [ 37 ]. This system generates possible differential diagnoses by selecting several symptoms or signs from a database that can be searched using a search box.…”

Utilizing Learn-to-Rank Systems for More Effective Diagnosis in Rural Family Medicine

“…However, the effectiveness of these diagnosis-supporting systems is not universal, as they are closely intertwined with the cultural and contextual nuances of different regions and populations [3]. The need for culturally sensitive and contextually appropriate systems is paramount, ensuring they are adaptable and resonate with the diverse patient demographics encountered by family physicians.…”

“…Integrating Japanese learn-to-rank approaches into the diagnostic process significantly aligns technological advancements with physicians' nuanced iterative strategies for differential diagnosis [3]. These approaches, precisely the listwise method, mirror the physicians' workflow of recalling, refining, and ranking multiple differential diseases, thereby fostering a symbiotic relationship between machine-learning frameworks and medical expertise.…”

“…A pivotal aspect of this integration is its potential to address existing technological gaps [3]. Current technologies are proficient in text-mining information related to confirmed diseases but fall short in extracting and converting data on multiple differential diseases from a vast body of medical literature.…”

“…Moreover, the significance of these approaches is underscored by the urgent need to enhance the predictive performance of clinical decision support systems (CDSS) [3]. By applying strict criteria focused on rare and difficult-to-diagnose cases, these approaches promise to cater to real-world challenges encountered by internists and general practitioners.…”