Development and evaluation of uncertainty quantifying machine learning models to predict piperacillin plasma concentrations in critically ill patients

Abstract

Background Beta-lactam antimicrobial concentrations are frequently suboptimal in critically ill patients. Population pharmacokinetic (PopPK) modeling is the golden standard to predict drug concentrations. However, currently available PopPK models often lack predictive accuracy, making them less suited to guide dosing regimen adaptations. Furthermore, many currently developed models for clinical applications often lack uncertainty quantification. We, therefore, aimed to develop machine learning (ML) models for the prediction of piperacillin plasma concentrations while also providing uncertainty quantification with the aim of clinical practice. Methods Blood samples for piperacillin analysis were prospectively collected from critically ill patients receiving continuous infusion of piperacillin/tazobactam. Interpretable ML models for the prediction of piperacillin concentrations were designed using CatBoost and Gaussian processes. Distribution-based Uncertainty Quantification was added to the CatBoost model using a proposed Quantile Ensemble method, useable for any model optimizing a quantile function. These models are subsequently evaluated using the distribution coverage error, a proposed interpretable uncertainty quantification calibration metric. Development and internal evaluation of the ML models were performed on the Ghent University Hospital database (752 piperacillin concentrations from 282 patients). Ensuing, ML models were compared with a published PopPK model on a database from the University Medical Centre of Groningen where a different dosing regimen is used (46 piperacillin concentrations from 15 patients.). Results The best performing model was the Catboost model with an RMSE and $$R^2$$ R 2 of 31.94–0.64 and 33.53–0.60 for internal evaluation with and without previous concentration. Furthermore, the results prove the added value of the proposed Quantile Ensemble model in providing clinically useful individualized uncertainty predictions and show the limits of homoscedastic methods like Gaussian Processes in clinical applications. Conclusions Our results show that ML models can consistently estimate piperacillin concentrations with acceptable and high predictive accuracy when identical dosing regimens as in the training data are used while providing highly relevant uncertainty predictions. However, generalization capabilities to other dosing schemes are limited. Notwithstanding, incorporating ML models in therapeutic drug monitoring programs seems definitely promising and the current work provides a basis for validating the model in clinical practice.

Machine Learning: A New Approach for Dose Individualization

“…After removing duplicated articles, 3,346 studies were screened by the title and/or abstract, 3,175 irrelevant studies were excluded and 171 articles were included for full‐text review. Finally, 64 articles related to precision dosing using ML were included for analysis 11–74 . The PRISMA flow diagram representing the study selection process and review results is presented in Figure .…”

Knowledge mapping of graph neural networks for drug discovery: a bibliometric and visualized analysis

“…The combination with transformer networks is also a recently popular combination, which can preserve the original information about the interactions between atoms in the chemical structure of a drug, overcoming the problem of a lack of learning of edge features by the graph convolutional neural network (Zhang et al, 2022c). In addition, improving the explainability of models is a common challenge for current machine learning models (Karimi et al, 2021;Verhaeghe et al, 2022). Existing methods to improve the interpretability of GNN are to introduce an attention mechanism (Karimi et al, 2021;Yang et al, 2022a), Yang's team (Karimi et al, 2019) developed the Deep Affinity model by introducing an attention mechanism based on a unified RNN-CNN.…”

Machine learning in the prediction and detection of new-onset atrial fibrillation in ICU: a systematic review

“…A comprehensive dataset comprising medical records from a total of 108,724 subjects was sourced from various databases, including Medical Information Mart for Intensive Care (MIMIC)-II [19], -III [20,21], -IV [22], Kensington General Hospital (KGH) database [23], Amster-damUMCdb [22], Ghent University Hospital database [22], and Belgian hospital ICU database [22]. The ICU admission records were retrieved for ML models design, testing, and external validation [22,23]. The records of ICU admissions used in the modeling studies spanned the period from 2001 to 2020 [19][20][21][22][23].…”