Automated En Masse Machine Learning Model Generation Shows Comparable Performance as Classic Regression Models for Predicting Delayed Graft Function in Renal Allografts

Abstract

Background. Several groups have previously developed logistic regression models for predicting delayed graft function (DGF). In this study, we used an automated machine learning (ML) modeling pipeline to generate and optimize DGF prediction models en masse. Methods. Deceased donor renal transplants at our institution from 2010 to 2018 were included. Input data consisted of 21 donor features from United Network for Organ Sharing. A training set composed of ~50%/50% split in DGF-positive and DGF-negative cases was used to generate 400 869 models. Each model was based on 1 of 7 ML algorithms (gradient boosting machine, k-nearest neighbor, logistic regression, neural network, naive Bayes, random forest, support vector machine) with various combinations of feature sets and hyperparameter values. Performance of each model was based on a separate secondary test dataset and assessed by common statistical metrics. Results. The best performing models were based on neural network algorithms, with the highest area under the receiver operating characteristic curve of 0.7595. This model used 10 out of the original 21 donor features, including age, height, weight, ethnicity, serum creatinine, blood urea nitrogen, hypertension history, donation after cardiac death status, cause of death, and cold ischemia time. With the same donor data, the highest area under the receiver operating characteristic curve for logistic regression models was 0.7484, using all donor features. Conclusions. Our automated en masse ML modeling approach was able to rapidly generate ML models for DGF prediction. The performance of the ML models was comparable with classic logistic regression models.

Artificial Intelligence—A Tool for Risk Assessment of Delayed-Graft Function in Kidney Transplant

“…In our work, we checked two machine learning techniques through by random forest and artificial neural network models. Similar models were described by other authors [20][21][22][23]. Neural networks aimed at assessing the occurrence of DGF achieved various performances, but clearly indicated the practicality and clinical usefulness of modern computer techniques.…”

“…There are many machine learning techniques, and models that are theoretically simpler and less complex can perform significantly better than advanced ones. Artificial neural networks can achieve good performance with AUROC 0.732 or AUROC 0.7595 [21,23], but a simpler model based on linear SVM achieves an AUROC of 0.843 [22]. The question is whether it is worth investing in the complexity of the model.…”

Prediction of tuberculosis using an automated machine learning platform for models trained on synthetic data

“…The same training and validation steps described above (in the non-automated RF approach) were also repeated through our automated machine learning approach through the Auto-ML platform MILO. As described previously,[ 17 18 19 ] the MILO platform incorporates an automated data processor, a data feature selector and data transformer, followed by multiple supervised ML model building approaches that make use of its custom hyperparameter search tools that help identify the optimal hyperparameter combinations for each of the seven algorithms utilized within MILO (neural network/multi-layer perceptron, logistic regression (LR), naïve Bayes (NB), k -nearest neighbor ( k- NN), support vector machine (SVM), random forest (RF), and XGBoost gradient boosting machine (GBM) techniques) [ Figure 3 ].…”

Machine learning in health care and laboratory medicine: General overview of supervised learning and Auto‐ML

“…12 The optimized ML model not only had >90% sensitivity and specificity rates but most notably were able to predict AKI 61.8 hours faster than the KDIGO criteria. 13 Similar approaches through such auto-ML have also produced models for delayed graft function (DGF), 14 an important clinical parameter within transplant patients as well as in certain COVID-19 testing platforms to develop predictive tools that embrace the MALDI-TOF platform. 15 In addition to the practical utility of the developed models discussed in these various common and critical clinical tasks, these studies also demonstrate how the appropriate auto-ML can be used to compare the relative value of features and therefore more quickly and empirically lead to the implementation of new laboratory assays which might otherwise take years to reach widespread adoption.…”