Development and validation of machine learning-driven prediction model for serious bacterial infection among febrile children in emergency departments

Abstract

Serious bacterial infection (SBI) in children, such as bacterial meningitis or sepsis, is an important condition that can lead to fatal outcomes. Therefore, since it is very important to accurately diagnose SBI, SBI prediction tools such as ‘Refined Lab-score’ or ‘clinical prediction rule’ have been developed and used. However, these tools can predict SBI only when there are values of all factors used in the tool, and if even one of them is missing, the tools become useless. Therefore, the purpose of this study was to develop and validate a machine learning-driven model to predict SBIs among febrile children, even with missing values. This was a multicenter retrospective observational study including febrile children <6 years of age who visited Emergency departments (EDs) of 3 different tertiary hospitals from 2016 to 2018. The SBI prediction model was trained with a derivation cohort (data from two hospitals) and externally tested with a validation cohort (data from a third hospital). A total of 11,973 and 2,858 patient records were included in the derivation and validation cohorts, respectively. In the derivation cohort, the area under the receiver operating characteristic curve (AUROC) of the RF model was 0.964 (95% confidence interval [CI], 0.943–0.986), and the area under the precision-recall curve (AUPRC) was 0.753 (95% CI, 0.681–0.824). The conventional LR (CLR) model showed corresponding values of 0.902 (95% CI, 0.894–0.910) and 0.573 (95% CI, 0.560–0.586), respectively. In the validation cohort, the AUROC (95% CI) of the RF model was 0.950 (95% CI, 0.945–0.956), the AUPRC was 0.605 (95% CI, 0.593–0.616), and the CLR presented corresponding values of 0.815 (95% CI, 0.789–0.841) and 0.586 (95% CI, 0.553–0.619), respectively. We developed a machine learning-driven prediction model for SBI among febrile children, which works robustly despite missing values. And it showed superior performance compared to CLR in both internal validation and external validation.

Febrile infants risk score at triage (FIRST) for the early identification of serious bacterial infections

“…A previous supervised learning model for risk stratification of febrile infants acknowledged that it lacked parameter cutoffs and was computationally complex 11 . While machine learning models have promising performance compared to traditional scoring systems 12 , 18 , these have been difficult to translate to clinical practice because of the lack of recommended thresholds for action. In contrast, we assigned risk scores that quantified risk for SBI at each predictive risk threshold.…”

Implications of Artificial Intelligence in Addressing Antimicrobial Resistance: Innovations, Global Challenges, and Healthcare’s Future

“…A study by Lee et al [ 50 ] demonstrated the feasibility of using machine learning models to predict bacterial or viral infections based on clinical symptoms and laboratory markers, thereby helping clinicians make informed antibiotic prescribing decisions. In addition to hospital-based applications, AI-DSS has the potential to extend antimicrobial stewardship efforts to community-based healthcare providers such as primary care physicians, urgent care clinics, and long-term care facilities [ 51 ].…”