Machine learning and synthetic outcome estimation for individualised antimicrobial cessation

Abstract

The decision on when it is appropriate to stop antimicrobial treatment in an individual patient is complex and under-researched. Ceasing too early can drive treatment failure, while excessive treatment risks adverse events. Under- and over-treatment can promote the development of antimicrobial resistance (AMR). We extracted routinely collected electronic health record data from the MIMIC-IV database for 18,988 patients (22,845 unique stays) who received intravenous antibiotic treatment during an intensive care unit (ICU) admission. A model was developed that utilises a recurrent neural network autoencoder and a synthetic control-based approach to estimate patients’ ICU length of stay (LOS) and mortality outcomes for any given day, under the alternative scenarios of if they were to stop vs. continue antibiotic treatment. Control days where our model should reproduce labels demonstrated minimal difference for both stopping and continuing scenarios indicating estimations are reliable (LOS results of 0.24 and 0.42 days mean delta, 1.93 and 3.76 root mean squared error, respectively). Meanwhile, impact days where we assess the potential effect of the unobserved scenario showed that stopping antibiotic therapy earlier had a statistically significant shorter LOS (mean reduction 2.71 days, p-value <0.01). No impact on mortality was observed. In summary, we have developed a model to reliably estimate patient outcomes under the contrasting scenarios of stopping or continuing antibiotic treatment. Retrospective results are in line with previous clinical studies that demonstrate shorter antibiotic treatment durations are often non-inferior. With additional development into a clinical decision support system, this could be used to support individualised antimicrobial cessation decision-making, reduce the excessive use of antibiotics, and address the problem of AMR.

Brave New World of Artificial Intelligence: Its Use in Antimicrobial Stewardship—A Systematic Review

“…A total of 4658 citations were identified from the three databases and, after removing the duplicates, 2839 were eligible for screening. A total of 1086 articles were assessed for eligibility and eighteen [20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37] were included in this systematic review (Figure 1). Most studies were excluded because they did not study the application of machine learning models nor their predictive performance or because they were not applied to hospital inpatients and outpatients with infections, such as studies in vitro or regarding drug development.…”

“…The number of features included in the machine learning algorithms ranged from 6 to 788. The patients included were from different settings; one (5.5%) study was designed for outpatients [35], and two were only applied to ICU patients [26,29]. The number of patients ranged from 48 (on a validation set) to 382,943.…”

“…All the studies were rated as being of "fair quality" by the NIH Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies; fourteen studies were rated as 57.1% and four [22,26,32,35] were rated as 64.3%. The participation rate, variation in amount or level of exposure, and loss to follow-up criteria were not applied to any of the studies.…”

Evaluating the impact of private land conservation with synthetic control design

“…There are also ways to control for confounding effects that could benefit conservation evaluations. Causal inference science is rapidly advancing with cutting-edge designs that use statistics, machine learning, and predictive modeling techniques to calculate average, conditional average (Athey & Wager, 2019), and individual-level impact estimates (Bolton et al, 2022), while offering greater potential for dealing with the confounding effects (Morgan & Winship, 2014;Sharma & Kiciman, 2020). One such powerful approach that is useful for estimating the impact of individual units (when time-series data are available) is synthetic control design.…”

Commentary: Machine learning in clinical decision-making

“…Clinical decision support systems have the potential to increase antimicrobial stewardship, thus mitigating antimicrobial resistance. Bolton et al use a machine learning and synthetic control-based approach to estimate patients’ length of stay (LOS) and mortality outcomes for any given day if they were to stop vs. continue antibiotic treatment ( 2 ). Comparisons between decision support system use and control experiences demonstrated minimal difference for both stopping and continuing scenarios, indicating that decision support estimations were reliable (average LOS differences of 0.24 and 0.42 days, respectively).…”