Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal

Abstract

Background The number of studies applying machine learning (ML) to predict acute kidney injury (AKI) has grown steadily over the last decade. We assess and critically appraise the state of the art in ML models for AKI prediction considering performance, methodological soundness, and applicability. Methods We searched PubMed and ArXiv, extracted data, and critically appraised studies based on the TRIPOD, CHARMS, and PROBAST guidelines. Results Forty-six studies from 3166 titles were included. Thirty-eight studies developed a model, five developed and externally validated one, and three studies externally validated one. Flexible ML methods were used more often than deep learning, although the latter was common with temporal variables and text as predictors. Predictive performance showed an area under receiver operating curves ranging from 0.49 to 0.99. Our critical appraisal identified a high risk of bias in thirty-nine studies. Some studies lacked internal validation, whereas external validation and interpretability of results were rarely considered. Fifteen studies focused on AKI prediction in the intensive care setting, and the USA-derived MIMIC dataset was commonly used. Reproducibility was limited as data and code were usually unavailable. Conclusions Flexible ML methods are popular for the prediction of AKI, although more complex models based on deep learning are emerging. Our critical appraisal identified a high risk of bias in most models: Studies should use calibration measures and external validation more often, improve model interpretability, and share data and code to improve reproducibility.

Machine Learning Decision Support Systems for Predicting Acute Kidney Injury: Improving Precision to improve patient outcomes

“…39 The DDA involves using statistical and computational techniques to obtain meaningful patterns and insights from complex datasets and using these ndings to inform the selection of appropriate ML algorithms, features, and hyperparameters. Although the MDA in uenced the ML development methodologies of most studies related to the prediction of AKI, 17,18 a combination of both architectures guided the project design and development of this study. The input features were identi ed based on literature review and consultation with kidney experts in our research team to guide the model developments.…”

“…12,15,16 The estimation of baseline sCr has been especially important in studies developing and validating machine learning (ML) models to predict AKI. 17,18 In general, these studies have used different methods of estimating the baseline sCr to establish the ground truth in order to label positive AKI occurrences. This can lead to discrepancies in how AKI events are identi ed and labelled, making direct comparisons of the models' performance metrics challenging.…”

“…The most recent study published in 2022, which reviewed the application of ML models to predict AKI, has indicated that only 25% of the included studies, reported the precision metric which evaluates the proportion of positive predictions made by the classi er that are correct. 17 Therefore, for the remaining studies, it is not possible to validate the false positive (FP) rates. An appraisal of studies with reported precision revealed a relatively high number of FP predictions.…”

Generalisability of AI-based scoring systems in the ICU: a systematic review and meta-analysis

“…The infrequent external validation of ML models for the prediction of acute events in the ICU was already noted in a 2019 systematic review, with only 7% of studies at the time using geographically independent data for model validation [8]. This has been echoed in more recent, disease-specific reviews looking at models for sepsis [20] and acute kidney injury [42]. While we showed that this percentage has somewhat improved since, we also find that challenges remain even if external validation is performed.…”

Data heterogeneity in federated learning with Electronic Health Records: Case studies of risk prediction for acute kidney injury and sepsis diseases in critical care

“…Both AKI and sepsis are also highly heterogeneous [21]. This makes models built with conventional FL strategies such as federated averaging challenging to generalize across clinics, limiting their use [7,22,23]. Several federated architectures have been proposed to mitigate effects of data heterogeneity in other domains and built personalized, but globally correlated, models to mitigate drift across sites [23], such as model-agnostic meta-learning (MAML), federated multitask learning, and knowledge distillation [24][25][26][27][28].…”