Training prediction models for individual risk assessment of postoperative complications after surgery for colorectal cancer

Lin, Viviane; Tsouchnika, Adamantia; Allakhverdiiev, Eldar; Rosen, Andreas Weinberger; Gögenur, Mikail; Bräuner, Karoline Bendix; Walbech, Julie Sparholt; Rijnbeek, Peter; Drakos, Iannis; Gögenür, Ismail

doi:10.1007/s10151-022-02624-x

Cited by 11 publications

(14 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The average AUROC for the best model across all 36 articles was 0.83; of the 8 articles with sample sizes of less than 2000 (ie, less than 1000 samples per class), 7 22,[26][27][28]31,35,37 (87.5%) had below-average (ie, less than 0.83) AUROC or accuracy. Twenty-four 9,18,20,[24][25][26][27][28][29][32][33][34][35]37,[39][40][41][43][44][45][46][47][48][49] articles (66.7%) presented confidence intervals for performance metrics. Among the 8 articles with sample sizes of less than 2000, 5 [26][27][28]35,37 presented confidence intervals.…”

Section: Resultsmentioning

confidence: 99%

“…Twentythree articles 9,19,21,[23][24][25]27,29,30,[32][33][34]36,37,39,41,42,[44][45][46][48][49][50] (63.9%) reported precision metrics (area under the precision-recall curve, positive predictive value, or F1 score). Twenty-five articles 9,20,21,[23][24][25][26][27][28]31,33,34,36,38,40,[42][43][44][45][46][47][48][49][50] (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs. Thirteen articles 9,16,17,…”

Section: Resultsmentioning

confidence: 99%

“…None of the articles described equity analyses in which model performance was stratified by sex or race. Twenty-three articles9,19,21,23–25,27,29,30,32–34,36,37,39,41,42,44–46,48–50 (63.9%) reported precision metrics (area under the precision-recall curve, positive predictive value, or F1 score). Twenty-five articles9,20,21,23–28,31,33,34,36,38,40,42–50 (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs.…”

Section: Resultsmentioning

confidence: 99%

“…Twenty-three articles9,19,21,23–25,27,29,30,32–34,36,37,39,41,42,44–46,48–50 (63.9%) reported precision metrics (area under the precision-recall curve, positive predictive value, or F1 score). Twenty-five articles9,20,21,23–28,31,33,34,36,38,40,42–50 (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs. Thirteen articles9,16,17,20,25,27,29,30,35,38,45,46,50 (36.1%) presented a framework that could be used for clinical implementation; none of the articles assessed the efficacy of clinical implementation.…”

Section: Resultsmentioning

confidence: 99%

See 3 more Smart Citations

Artificial Intelligence–enabled Decision Support in Surgery

et al. 2023

View full text Add to dashboard Cite

Objective: To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting. Background: To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown. Methods: Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Results: Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2000; 7 of these 8 articles (87.5%) had below-average (< 0.83) area under the receiver operating characteristic or accuracy. Overall, 29 articles (80.6%) performed internal validation only, 5 (13.8%) performed external validation, and 2 (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy. Conclusions: Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Artificial Intelligence–enabled Decision Support in Surgery

et al. 2023

View full text Add to dashboard Cite

show abstract

“…We trained a Least Absolute Shrinkage and Selection Operator (LASSO) Logistic Regression model for 30-day, 90day, and 1-year postoperative mortality [29][30][31][32][33]. The LASSO Logistic Regression generally performs well with rare outcomes or sparse data.…”

Section: Statistical Analysesmentioning

confidence: 99%

Prediction of 30-day, 90-day and 1 year mortality after colorectal cancer surgery using a data-driven approach

Bräuner,

Tsouchnika,

Mashkoor

et al. 2023

Preprint

View full text Add to dashboard Cite

Purpose: To develop prediction models for short-term mortality risk assessment following colorectal cancer surgery. Background: Individual risk prediction for mortality following CRC surgery can assist surgeons in clinical decision-making and patient-centered care. Several risk scoring systems exist to assess the risk of postoperative mortality following a CRC resection, but they are based on intra- or postoperative variables or do not represent contemporary clinical practice. Methods: Data was harmonized from four Danish observational health databases into the Observational Medical Outcomes Partnership Common Data Model. With a data-driven approach using the Least Absolute Shrinkage and Selection Operator logistic regression on preoperative data, we developed 30-day, 90-day, and 1-year mortality prediction models. We assessed discriminative performance using the area under the receiver operating characteristic and precision-recall curve and calibration using calibration slope, intercept, and calibration-in-the-large. We additionally assessed model performance in subgroups of curative, palliative, elective, and emergency surgery. Results: A total of 57,558 patients were included in the study population. The model showed good discrimination with an area under the receiver-operating characteristic curve of 0.88, 0.878, and 0.861 for 30-day, 90-day, and 1-year mortality respectively, and a calibration-in-the-large of 1.01, 0.99, and 0.99. Conclusion: We were able to train prediction models for the risk of short-term mortality on a data set of four combined national health databases with good discrimination and calibration.

show abstract

Prediction of 30-day, 90-day, and 1-year mortality after colorectal cancer surgery using a data-driven approach

Bräuner,

Tsouchnika,

Mashkoor

et al. 2024

Int J Colorectal Dis

View full text Add to dashboard Cite

Purpose To develop prediction models for short-term mortality risk assessment following colorectal cancer surgery. Methods Data was harmonized from four Danish observational health databases into the Observational Medical Outcomes Partnership Common Data Model. With a data-driven approach using the Least Absolute Shrinkage and Selection Operator logistic regression on preoperative data, we developed 30-day, 90-day, and 1-year mortality prediction models. We assessed discriminative performance using the area under the receiver operating characteristic and precision-recall curve and calibration using calibration slope, intercept, and calibration-in-the-large. We additionally assessed model performance in subgroups of curative, palliative, elective, and emergency surgery. Results A total of 57,521 patients were included in the study population, 51.1% male and with a median age of 72 years. The model showed good discrimination with an area under the receiver operating characteristic curve of 0.88, 0.878, and 0.861 for 30-day, 90-day, and 1-year mortality, respectively, and a calibration-in-the-large of 1.01, 0.99, and 0.99. The overall incidence of mortality were 4.48% for 30-day mortality, 6.64% for 90-day mortality, and 12.8% for 1-year mortality, respectively. Subgroup analysis showed no improvement of discrimination or calibration when separating the cohort into cohorts of elective surgery, emergency surgery, curative surgery, and palliative surgery. Conclusion We were able to train prediction models for the risk of short-term mortality on a data set of four combined national health databases with good discrimination and calibration. We found that one cohort including all operated patients resulted in better performing models than cohorts based on several subgroups.

show abstract

Training prediction models for individual risk assessment of postoperative complications after surgery for colorectal cancer

Cited by 11 publications

References 35 publications

Artificial Intelligence–enabled Decision Support in Surgery

Artificial Intelligence–enabled Decision Support in Surgery

Prediction of 30-day, 90-day and 1 year mortality after colorectal cancer surgery using a data-driven approach

Prediction of 30-day, 90-day, and 1-year mortality after colorectal cancer surgery using a data-driven approach

Contact Info

Product

Resources

About