A Web-Based Calculator to Predict Early Death Among Patients With Bone Metastasis Using Machine Learning Techniques: Development and Validation Study

Abstract

Background Patients with bone metastasis often experience a significantly limited survival time, and a life expectancy of <3 months is generally regarded as a contraindication for extensive invasive surgeries. In this context, the accurate prediction of survival becomes very important since it serves as a crucial guide in making clinical decisions. Objective This study aimed to develop a machine learning–based web calculator that can provide an accurate assessment of the likelihood of early death among patients with bone metastasis. Methods This study analyzed a large cohort of 118,227 patients diagnosed with bone metastasis between 2010 and 2019 using the data obtained from a national cancer database. The entire cohort of patients was randomly split 9:1 into a training group (n=106,492) and a validation group (n=11,735). Six approaches—logistic regression, extreme gradient boosting machine, decision tree, random forest, neural network, and gradient boosting machine—were implemented in this study. The performance of these approaches was evaluated using 11 measures, and each approach was ranked based on its performance in each measure. Patients (n=332) from a teaching hospital were used as the external validation group, and external validation was performed using the optimal model. Results In the entire cohort, a substantial proportion of patients (43,305/118,227, 36.63%) experienced early death. Among the different approaches evaluated, the gradient boosting machine exhibited the highest score of prediction performance (54 points), followed by the neural network (52 points) and extreme gradient boosting machine (50 points). The gradient boosting machine demonstrated a favorable discrimination ability, with an area under the curve of 0.858 (95% CI 0.851-0.865). In addition, the calibration slope was 1.02, and the intercept-in-large value was −0.02, indicating good calibration of the model. Patients were divided into 2 risk groups using a threshold of 37% based on the gradient boosting machine. Patients in the high-risk group (3105/4315, 71.96%) were found to be 4.5 times more likely to experience early death compared with those in the low-risk group (1159/7420, 15.62%). External validation of the model demonstrated a high area under the curve of 0.847 (95% CI 0.798-0.895), indicating its robust performance. The model developed by the gradient boosting machine has been deployed on the internet as a calculator. Conclusions This study develops a machine learning–based calculator to assess the probability of early death among patients with bone metastasis. The calculator has the potential to guide clinical decision-making and improve the care of patients with bone metastasis by identifying those at a higher risk of early death.

Interpretability-based machine learning for predicting the risk of death from pulmonary inflammation in Chinese intensive care unit patients

“…SHAP allows for a global evaluation of the model by determining the marginal contribution of features to the model output. Complementing the SHAP method, the Local Interpretable Model-Agnostic Explanations (LIME) method improves the interpretability of the best model and its transparency in clinical practice ( 19 ). LIME calculates the risk of premature death and assigns individual weights to each variable, helping to understand changes in estimated probabilities under different observation settings and making the model more distinct.…”

Presentation_1.pdf

“…SHAP allows for a global evaluation of the model by determining the marginal contribution of features to the model output. Complementing the SHAP method, the Local Interpretable Model-Agnostic Explanations (LIME) method improves the interpretability of the best model and its transparency in clinical practice (19). LIME calculates the risk of premature death and assigns individual weights to each variable, helping to understand changes in estimated probabilities under different observation settings and making the model more distinct.…”