Development and evaluation of machine learning models and nomogram for the prediction of severe acute pancreatitis

Abstract

Background and Aim Severe acute pancreatitis (SAP) in patients progresses rapidly and can cause multiple organ failures associated with high mortality. We aimed to train a machine learning (ML) model and establish a nomogram that could identify SAP, early in the course of acute pancreatitis (AP). Methods In this retrospective study, 631 patients with AP were enrolled in the training cohort. For predicting SAP early, five supervised ML models were employed, such as random forest (RF), K‐nearest neighbors (KNN), and naive Bayes (NB), which were evaluated by accuracy (ACC) and the areas under the receiver operating characteristic curve (AUC). The nomogram was established, and the predictive ability was assessed by the calibration curve and AUC. They were externally validated by an independent cohort of 109 patients with AP. Results In the training cohort, the AUC of RF, KNN, and NB models were 0.969, 0.954, and 0.951, respectively, while the AUC of the Bedside Index for Severity in Acute Pancreatitis (BISAP), Ranson and Glasgow scores were only 0.796, 0.847, and 0.837, respectively. In the validation cohort, the RF model also showed the highest AUC, which was 0.961. The AUC for the nomogram was 0.888 and 0.955 in the training and validation cohort, respectively. Conclusions Our findings suggested that the RF model exhibited the best predictive performance, and the nomogram provided a visual scoring model for clinical practice. Our models may serve as practical tools for facilitating personalized treatment options and improving clinical outcomes through pre‐treatment stratification of patients with AP.

Predictive value of machine learning for the severity of acute pancreatitis: A systematic review and meta-analysis

“…These newly identified studies were screened based on the inclusion and exclusion criteria. Finally, a total of 33 original studies [ [14] , [15] , [16] , [17] , [18] , [19] , [20] , [21] , [22] , [23] , [24] , [25] , [26] , [27] , [28] , [29] , [30] , [31] , [32] , [33] , [34] , [35] , [36] , [37] , [38] , [39] , [40] , [41] , [42] , [43] , [44] , [45] , [46] ] were included. The literature screening process is illustrated in Fig.…”

“…These studies were conducted in 8 countries, including 1 [ 30 ] in Germany, 1 [ 26 ] in Hungary, 1 [ 17 ] in South Korea, 1 [ 37 ] in Nepal, 1 [ 18 ] in Sweden, 2 [ 20 , 21 ] in UK, 3 [ 22 , 28 , 43 ] in the USA, with the remaining studies conducted in China. Ten [ 23 , 24 , 26 , 27 , 30 , 31 , 34 , 35 , 40 , 41 ] studies were multicenter studies, while two studies [ 18 , 20 ] collected subjects from databases. Eleven studies [ [14] , [15] , [16] , 18 , [20] , [21] , [22] , 25 , 26 , 31 , 35 ] considered overfitting, and k-fold cross-validation was primarily used.…”

“…In terms of statistical methods, ten studies [ 17 , 18 , 25 , 26 , [39] , [40] , [41] , 44 , 45 ] had an EPV <10 in the training set, and nine studies [ [20] , [21] , [22] , 24 , 30 , 33 , 34 , 36 , 46 ] were unable to calculate EPV. In addition, eleven studies [ 14 , 16 , 17 , 21 , 22 , 24 , 29 , 30 , 36 , 37 , 39 ] had no independent validation sets or had independent validation sets with fewer than 100 cases.…”

Acute pancreatitis: A review of diagnosis, severity prediction and prognosis assessment from imaging technology, scoring system and artificial intelligence

“…In a retrospective analysis involving 648 AP patients, Hong et al [ 158 ] developed RF and logistic regression models using a training sample; the RF model, notable for its interpretability, showcased the most superior discriminative performance in predicting SAP. In a retrospective study involving 631 AP patients, Luo et al [ 159 ] developed a machine learning model, culminating in a nomogram designed for the early identification of SAP during the progression of AP. Their findings indicated that the RF model delivered optimal predictive performance, with the nomogram offering a visual scoring model suitable for clinical application[ 159 ].…”

“…In a retrospective study involving 631 AP patients, Luo et al [ 159 ] developed a machine learning model, culminating in a nomogram designed for the early identification of SAP during the progression of AP. Their findings indicated that the RF model delivered optimal predictive performance, with the nomogram offering a visual scoring model suitable for clinical application[ 159 ]. Such models have the potential to act as functional tools, enabling personalized treatment choices and enhancing clinical results by stratifying AP patients prior to treatment[ 159 ].…”