International external validation of the SORG machine learning algorithms for predicting 90-day and one-year survival of patients with spine metastases using a Taiwanese cohort

Yang, Jyh‐Chin; Chen, Chih-Wei; Fourman, Mitchell S.; Bongers, Michiel E R; Karhade, Aditya V.; Groot, Olivier Q.; Lin, Wei-Hsin; Yen, Hung-Kuan; Huang, Pei-Ming; Yang, Sen; Schwab, Joseph H.; Hu, Ming-Hsiao

doi:10.1016/j.spinee.2021.01.027

Cited by 40 publications

(39 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This external validation cohort constitutes the first European test of the SORG ML and SORG nomogram predictive value and shows promising results that shall encourage the multidisciplinary teams to use these tools in a daily basis. Similar findings were already published in a Taiwainese cohort [31]. These algorithms showed a performance better than the currently utilized Tokuhashi revised, being more sensitive and specific in estimating 3-and 12-months survival in patients with metastatic spine disease.…”

Section: Discussionsupporting

confidence: 83%

SORG algorithm to predict 3- and 12-month survival in metastatic spinal disease: a cross-sectional population-based retrospective study

et al. 2022

View full text Add to dashboard Cite

In this study, we recruited 40 patients with spinal metastatic disease who were operated at Geneva University Hospitals by the Neurosurgery or Orthopedic teams between the years of 2015 and 2020. We did an ROC analysis in order to determine the accuracy of the SORG ML algorithm and nomogram versus the Tokuhashi Original and Revised Scores.The analysis of data of our independent cohort shows a clear advantage in terms of predictive ability of the SORG ML algorithm and nomogram in comparison with the Tokuhashi scores. The SORG ML had an AUC of 0.87 for 90-days and 0.85 for 1-year. The SORG Nomogram showed a predictive ability at 90-days and 1-year with AUC's of 0.87 and 0.76 respectively.These results showed excellent discriminative ability as compared with the Tokuhashi Original Score which achieved AUC's of 0.70 and 0.69 and the Tokuhashi Revised Score which had AUC's of 0.65 and 0.71 for 3-month and 1-year respectively. The predictive ability of the SORG ML algorithm and nomogram were superior to currently used preoperative survival estimation scores for spinal metastatic disease.

show abstract

Section: Discussionsupporting

confidence: 83%

SORG algorithm to predict 3- and 12-month survival in metastatic spinal disease: a cross-sectional population-based retrospective study

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Moreover, recently, a systemic review revealed that SORG Nomogram and machine learning algorithms showed superior performance in survival prediction for surgery in spinal metastases. 31 32 33) However, further improvement by comparative validation in large multicenter, prospective cohorts can still be obtained. Moreover, recently Karhade et al 34) developed machine learning algorithms for prediction of mortality after surgery for spinal metastasis since 2019.…”

Section: Discussionmentioning

confidence: 99%

Validation of Traditional Prognosis Scoring Systems and Skeletal Oncology Research Group Nomogram for Predicting Survival of Spinal Metastasis Patients Undergoing Surgery

Chanplakorn

Jaipanya

et al. 2022

Clin Orthop Surg

View full text Add to dashboard Cite

Background Many scoring systems that predict overall patient survival are based on clinical parameters and primary tumor type. To date, no consensus exists regarding which scoring system has the greatest predictive survival accuracy, especially when applied to specific primary tumors. Additionally, such scores usually fail to include modern treatment modalities, which influence patient survival. This study aimed to evaluate both the overall predictive accuracy of such scoring systems and the predictive accuracy based on the primary tumor. Methods A retrospective review on spinal metastasis patients who were aged more than 18 years and underwent surgical treatment was conducted between October 2008 and August 2018. Patients were scored based on data before the time of surgery. A survival probability was calculated for each patient using the given scoring systems. The predictive ability of each scoring system was assessed using receiver operating characteristic analysis at postoperative time points; area under the curve was then calculated to quantify predictive accuracy. Results A total of 186 patients were included in this analysis: 101 (54.3%) were men and the mean age was 57.1 years. Primary tumors were lung in 37 (20%), breast in 26 (14%), prostate in 20 (10.8%), hematologic malignancy in 18 (9.7%), thyroid in 10 (5.4%), gastrointestinal tumor in 25 (13.4%), and others in 40 (21.5%). The primary tumor was unidentified in 10 patients (5.3%). The overall survival was 201 days. For survival prediction, the Skeletal Oncology Research Group (SORG) nomogram showed the highest performance when compared to other prognosis scores in all tumor metastasis but a lower performance to predict survival with lung cancer. The revised Katagiri score demonstrated acceptable performance to predict death for breast cancer metastasis. The Tomita and revised Tokuhashi scores revealed acceptable performance in lung cancer metastasis. The New England Spinal Metastasis Score showed acceptable performance for predicting death in prostate cancer metastasis. SORG nomogram demonstrated acceptable performance for predicting death in hematologic malignancy metastasis at all time points. Conclusions The results of this study demonstrated inconsistent predictive performance among the prediction models for the specific primary tumor types. The SORG nomogram revealed the highest predictive performance when compared to previous survival prediction models.

show abstract

“…The following preoperative data were extracted: age at the time of surgery, sex, preoperative hemoglobin concentration (g/dL), absolute lymphocyte count (k/uL), the presence of visceral and lymph node metastases, impending or completed pathologic fracture, number of bone metastases, the ECOG score, and the patient's primary tumor type (7,8). The surgeon's estimation of survival was omitted since this was not recorded in the electronic medical records.…”

Section: Prognostic Variables and Outcomementioning

confidence: 99%

“…A slight decline in MC was observed when the 6-month and 12-month predictions were compared with the 12-month (MC = 0.86) and 18-month predictions (MC = 0.88), respectively. Since inconsistencies could happen with any survival prediction models, it is important to interpret their estimations onstrated that a machine-learning model's performance could vary in ethnogeographically distinct populations (8,16) and repeated validation in different cohorts is needed (9,15).…”

Section: Model Consistency Of Pathfxmentioning

confidence: 99%

“…It has also been recently updated to the 3rd version (3) to provide predictions for patients treated both operatively and with radiation only. However, some studies suggested PSSs could perform differently between ethnogeographically distinct cohorts (4,5,(8)(9)(10)(11)(12), and they should have been validated before being applied onto a specific population. The PATHFx offers survival predictions at 6 different time points: 1, 3, 6, 12, 18, and 24 months.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Can a Bayesian belief network for survival prediction in patients with extremity metastases (PATHFx) be externally validated in an Asian cohort of 356 surgically treated patients?

Hsieh

Lai

Lee

et al. 2022

ActaO

Self Cite

View full text Add to dashboard Cite

Background and purpose: Predicted survival may influence the treatment decision for patients with skeletal extremity metastasis, and PATHFx was designed to predict the likelihood of a patient dying in the next 24 months. However, the performance of prediction models could have ethnogeographical variations. We asked if PATHFx generalized well to our Taiwanese cohort consisting of 356 surgically treated patients with extremity metastasis.Patients and methods: We included 356 patients who underwent surgery for skeletal extremity metastasis in a tertiary center in Taiwan between 2014 and 2019 to validate PATHFx’s survival predictions at 6 different time points. Model performance was assessed by concordance index (c-index), calibration analysis, decision curve analysis (DCA), Brier score, and model consistency (MC).Results: The c-indexes for the 1-, 3-, 6-, 12-, 18-, and 24-month survival estimations were 0.71, 0.66, 0.65, 0.69, 0.68, and 0.67, respectively. The calibration analysis demonstrated positive calibration intercepts for survival predictions at all 6 timepoints, indicating PATHFx tended to underestimate the actual survival. The Brier scores for the 6 models were all less than their respective null model’s. DCA demonstrated that only the 6-, 12-, 18-, and 24-month predictions appeared useful for clinical decision-making across a wide range of threshold probabilities. The MC was < 0.9 when the 6- and 12-month models were compared with the 12-month and 18-month models, respectively.Interpretation: In this Asian cohort, PATHFx’s performance was not as encouraging as those of prior validation studies. Clinicians should be cognizant of the potential decline in validity of any tools designed using data outside their particular patient population. Developers of survival prediction tools such as PATHFx might refine their algorithms using data from diverse, contemporary patients that is more reflective of the world’s population.

show abstract

International external validation of the SORG machine learning algorithms for predicting 90-day and one-year survival of patients with spine metastases using a Taiwanese cohort

Cited by 40 publications

References 35 publications

SORG algorithm to predict 3- and 12-month survival in metastatic spinal disease: a cross-sectional population-based retrospective study

SORG algorithm to predict 3- and 12-month survival in metastatic spinal disease: a cross-sectional population-based retrospective study

Validation of Traditional Prognosis Scoring Systems and Skeletal Oncology Research Group Nomogram for Predicting Survival of Spinal Metastasis Patients Undergoing Surgery

Can a Bayesian belief network for survival prediction in patients with extremity metastases (PATHFx) be externally validated in an Asian cohort of 356 surgically treated patients?

Contact Info

Product

Resources

About