Automated Machine Learning for the Early Prediction of the Severity of Acute Pancreatitis in Hospitals

Abstract

BackgroundMachine learning (ML) algorithms are widely applied in building models of medicine due to their powerful studying and generalizing ability. This study aims to explore different ML models for early identification of severe acute pancreatitis (SAP) among patients hospitalized for acute pancreatitis.MethodsThis retrospective study enrolled patients with acute pancreatitis (AP) from multiple centers. Data from the First Affiliated Hospital and Changshu No. 1 Hospital of Soochow University were adopted for training and internal validation, and data from the Second Affiliated Hospital of Soochow University were adopted for external validation from January 2017 to December 2021. The diagnosis of AP and SAP was based on the 2012 revised Atlanta classification of acute pancreatitis. Models were built using traditional logistic regression (LR) and automated machine learning (AutoML) analysis with five types of algorithms. The performance of models was evaluated by the receiver operating characteristic (ROC) curve, the calibration curve, and the decision curve analysis (DCA) based on LR and feature importance, SHapley Additive exPlanation (SHAP) Plot, and Local Interpretable Model Agnostic Explanation (LIME) based on AutoML.ResultsA total of 1,012 patients were included in this study to develop the AutoML models in the training/validation dataset. An independent dataset of 212 patients was used to test the models. The model developed by the gradient boost machine (GBM) outperformed other models with an area under the ROC curve (AUC) of 0.937 in the validation set and an AUC of 0.945 in the test set. Furthermore, the GBM model achieved the highest sensitivity value of 0.583 among these AutoML models. The model developed by eXtreme Gradient Boosting (XGBoost) achieved the highest specificity value of 0.980 and the highest accuracy of 0.958 in the test set.ConclusionsThe AutoML model based on the GBM algorithm for early prediction of SAP showed evident clinical practicability.

Predictive value of machine learning for the severity of acute pancreatitis: A systematic review and meta-analysis

“…These newly identified studies were screened based on the inclusion and exclusion criteria. Finally, a total of 33 original studies [ [14] , [15] , [16] , [17] , [18] , [19] , [20] , [21] , [22] , [23] , [24] , [25] , [26] , [27] , [28] , [29] , [30] , [31] , [32] , [33] , [34] , [35] , [36] , [37] , [38] , [39] , [40] , [41] , [42] , [43] , [44] , [45] , [46] ] were included. The literature screening process is illustrated in Fig.…”

“…These studies were conducted in 8 countries, including 1 [ 30 ] in Germany, 1 [ 26 ] in Hungary, 1 [ 17 ] in South Korea, 1 [ 37 ] in Nepal, 1 [ 18 ] in Sweden, 2 [ 20 , 21 ] in UK, 3 [ 22 , 28 , 43 ] in the USA, with the remaining studies conducted in China. Ten [ 23 , 24 , 26 , 27 , 30 , 31 , 34 , 35 , 40 , 41 ] studies were multicenter studies, while two studies [ 18 , 20 ] collected subjects from databases. Eleven studies [ [14] , [15] , [16] , 18 , [20] , [21] , [22] , 25 , 26 , 31 , 35 ] considered overfitting, and k-fold cross-validation was primarily used.…”

“…Ten [ 23 , 24 , 26 , 27 , 30 , 31 , 34 , 35 , 40 , 41 ] studies were multicenter studies, while two studies [ 18 , 20 ] collected subjects from databases. Eleven studies [ [14] , [15] , [16] , 18 , [20] , [21] , [22] , 25 , 26 , 31 , 35 ] considered overfitting, and k-fold cross-validation was primarily used. The original studies collectively constructed 55 new machine-learning models and evaluated three primary clinical scales: APACHE II, BISAP, and Ranson.…”

Acute pancreatitis: A review of diagnosis, severity prediction and prognosis assessment from imaging technology, scoring system and artificial intelligence

“…Such models have the potential to act as functional tools, enabling personalized treatment choices and enhancing clinical results by stratifying AP patients prior to treatment[ 159 ]. In a study with a total of 1012 patients, Yin et al [ 160 ] developed a series of effective models for early prediction of SAP based on automated machine learning (AutoML) platform, and these models outperformed the existing scoring systems, which might offer insights into AutoML applications in future medical studies. The AutoML model based on the GBM algorithm for early prediction of SAP showed evident clinical practicability[ 160 ].…”

Construction and validation of machine learning models for sepsis prediction in patients with acute pancreatitis

“…A study conducted in the Chinese population used a machine learning model for accurate prediction of sepsis in intensive care unit (ICU) patients, the established machine learning-based model showed good predictive ability with AUC being 0.91 [ 16 ]. In addition, the machine learning model also showed excellent predictive value for severe AP and concurrent acute kidney injury (AKI) risk in AP [ 17 , 18 ]. However, to the best of our knowledge, no study has reported the application of machine learning in predicting the risk of sepsis in patients with AP.…”