Background Acute kidney injury (AKI) is commonly encountered in clinical practice and is associated with poor patient outcomes and increased health care costs. Despite it posing significant challenges for clinicians, effective measures for AKI prediction and prevention are lacking. Previously published AKI prediction models mostly have a simple design without external validation. Furthermore, little is known about the process of linking model output and clinical decisions due to the black-box nature of neural network models. Objective We aimed to present an externally validated recurrent neural network (RNN)–based continuous prediction model for in-hospital AKI and show applicable model interpretations in relation to clinical decision support. Methods Study populations were all patients aged 18 years or older who were hospitalized for more than 48 hours between 2013 and 2017 in 2 tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographic data, laboratory values, vital signs, and clinical conditions of patients were obtained from electronic health records of each hospital. We developed 2-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicted the future trajectory of creatinine values up to 72 hours. The performance of each developed model was evaluated using the internal and external validation data sets. For the explainability of our models, different model-agnostic interpretation methods were used, including Shapley Additive Explanations, partial dependence plots, individual conditional expectation, and accumulated local effects plots. Results We included 69,081 patients in the training, 7675 in the internal validation, and 72,352 in the external validation cohorts for model development after excluding cases with missing data and those with an estimated glomerular filtration rate less than 15 mL/min/1.73 m2 or end-stage kidney disease. Model 1 predicted any AKI development with an area under the receiver operating characteristic curve (AUC) of 0.88 (internal validation) and 0.84 (external validation), and stage 2 or higher AKI development with an AUC of 0.93 (internal validation) and 0.90 (external validation). Model 2 predicted the future creatinine values within 3 days with mean-squared errors of 0.04-0.09 for patients with higher risks of AKI and 0.03-0.08 for those with lower risks. Based on the developed models, we showed AKI probability according to feature values in total patients and each individual with partial dependence, accumulated local effects, and individual conditional expectation plots. We also estimated the effects of feature modifications such as nephrotoxic drug discontinuation on future creatinine levels. Conclusions We developed and externally validated a continuous AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts; thus, we suggest approaches to support clinical decisions based on prediction models for in-hospital AKI.

No abstract

BACKGROUND Acute kidney injury(AKI) is commonly encountered in clinical practice and associated with poor patient outcomes and increased healthcare costs. AKI poses significant challenges for clinicians but effective measures for the prediction and prevention of AKI are lacking. Previously published AKI prediction models mostly have simple design without external validation. Furthermore, little is known about how to link the model output and clinical decision supports due to the blackbox nature of the neural network models. OBJECTIVE We aimed to present an externally validated recurrent neural network (RNN)-based prediction model for in-hospital AKI, and to show the explainability of the model in relation to clinical decision support. METHODS Study populations were all patients aged ≥ 18 years and hospitalized more than a week from 2013 to 2017 in two tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographics, laboratory values, vital signs, and clinical conditions were obtained from the EHR of each hospital. A total of 102 variables included in the model. Each variable falls into two categories: static and dynamic variables. We developed two-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for Model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicts the future trajectory of Cr values up to 72 hours. Internal validation was performed by 5-fold cross validation using the training set, and then external validation was done using independent test set. RESULTS Of a total of 118,893 patients initially screened, after excluding cases with missing data and estimated glomerular filtration rate <15 ml/min/1.73m2 or end-stage kidney disease, 40,552 patients in training cohort and 4,084 in external validation cohort (test cohort) were used for model development. Model 1 with the observation window of 3 days predicts AKI development with the area under the curve of 0.80 (sensitivity 0.72, specificity 0.89) in external validation. The model 2 predicted the future creatinine values within 3 days with the mean square errors of 0.04-0.06 for patients with higher risks of AKI and 0.05-0.12 for those with lower risks. On the basis of the developed models, we showed the probability of AKI according to the feature values in total patients and each individual with partial dependence plots and individual conditional expectation plots. In addition, we estimated the effects of feature modifications such as nephrotoxic drug discontinuation on the future creatinine levels. CONCLUSIONS We developed and externally validated a real-time AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts. These suggest approaches to support clinical decisions based on the prediction models for in-hospital AKI.

No abstract