Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data

Wagner, Martin; Brandenburg, Johanna M.; Bodenstedt, Sebastian; Schulze, A.; Jenke, Alexander; Stern, Antonia; Daum, Marie T. J.; Mündermann, Lars; Kolbinger, Fiona R.; Bhasker, Nithya; Schneider, Gerd; Krause-Jüttler, Grit; Alwanni, Hisham; Fritz-Kebede, Fleur; Burgert, Oliver; Wilhelm, Dirk; Fallert, Johannes; Nickel, Felix; Maier-Hein, Lena; Dugas, Martin; Distler, Marius; Weitz, Jürgen; Müller-Stich, Beat-Peter; Speidel, Stefanie

doi:10.1007/s00464-022-09611-1

Cited by 14 publications

(12 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most accurate models will likely incorporate all available clinical, endoscopic, radiologic, and genetic information in a comprehensive evaluation. 46…”

Section: Discussionmentioning

confidence: 99%

“…The most accurate models will likely incorporate all available clinical, endoscopic, radiologic, and genetic information in a comprehensive evaluation. 46 Our study has multiple limitations. First, we did not include patients treated nonoperatively after a complete clinical response, as the NCDB does not currently include posttreatment restaging or disease-free survival as an outcome.…”

Section: Microsatellite Instabilitymentioning

confidence: 93%

See 1 more Smart Citation

Prediction of Pathologic Complete Response for Rectal Cancer Based on Pre-treatment Factors Using Machine Learning

Chen,

Goffredo,

Butler

et al. 2023

Diseases of the Colon &Amp; Rectum

View full text Add to dashboard Cite

BACKGROUND: Pathologic complete response after neoadjuvant therapy is an important prognostic indicator for locally advanced rectal cancer and may give insights into which patients might be treated nonoperatively in the future. Existing models for predicting pathologic complete response in the pre-treatment setting are limited by small datasets and low accuracy. OBJECTIVE: We sought to use machine learning to develop a more generalizable predictive model for pathologic complete response for locally advanced rectal cancer. DESIGN: Patients with locally advanced rectal cancer who underwent neoadjuvant therapy followed by surgical resection were identified in the National Cancer Database from years 2010-2019 and were split into training, validation, and test sets. Machine learning techniques included random forest, gradient boosting, and artificial neural network. A logistic regression model was also created. Model performance was assessed using area under the receiver operating characteristic curve. SETTINGS: This study used a national, multicenter dataset. PATIENTS: Patients with locally advanced rectal cancer who underwent neoadjuvant therapy and proctectomy. MAIN OUTCOME MEASURES: Pathologic complete response defined as T0/xN0/x. RESULTS: The dataset included 53,684 patients. 22.9% of patients experienced pathologic complete response. Gradient boosting showed the best performance with area under the receiver operating characteristic curve of 0.777 (95% CI, 0.773 - 0.781), compared with 0.684 (95% CI, 0.68 - 0.688) for logistic regression. The strongest predictors of pathologic complete response were no lymphovascular invasion, no perineural invasion, lower CEA, smaller size of tumor, and microsatellite stability. A concise model including the top 5 variables showed preserved performance. LIMITATIONS: The models were not externally validated. CONCLUSIONS: Machine learning techniques can be used to accurately predict pathologic complete response for locally advanced rectal cancer in the pretreatment setting. After fine-tuning on a dataset including patients treated nonoperatively, these models could help clinicians identify the appropriate candidates for a watch and wait strategy

show abstract

“…The most accurate models will likely incorporate all available clinical, endoscopic, radiologic, and genetic information in a comprehensive evaluation. 46…”

Section: Discussionmentioning

confidence: 99%

Section: Microsatellite Instabilitymentioning

confidence: 93%

Prediction of Pathologic Complete Response for Rectal Cancer Based on Pre-treatment Factors Using Machine Learning

Chen,

Goffredo,

Butler

et al. 2023

Diseases of the Colon &Amp; Rectum

View full text Add to dashboard Cite

show abstract

“…The sum of these intraoperative process characteristics was recently summarized by the term surgomics. [43] In a further step, the intraoperative data can be automatically analyzed by AI. Surgomics can be used for intraoperative assistance such as image-guided surgery, decision support systems, real-time feedback with teaching and context-aware augmented reality.…”

Section: Intraoperative Managementmentioning

confidence: 99%

Artificial intelligence in pancreatic surgery: current applications

Kuemmerli¹,

Rössler²,

Berchtold³

et al. 2023

Journal of Pancreatology

Self Cite

View full text Add to dashboard Cite

Surgery plays a central role in the treatment of benign and malignant pancreatic diseases. Artificial intelligence (AI) is an important upcoming technology to support surgeons in pre-, intra-, and postoperative diagnosis, decision-making and training toward an optimized patient care. Current AI applications show a promising role in the evaluation of preoperative images for prediction of malignancy and resectability, intraoperative decision support, surgical training as well as a postoperative risk stratification to personalize the management of complications. This scoping review summarizes the most up to date developments of AI in pancreatic surgery with the highest available level of evidence.

show abstract

“…The concept of Surgomics uses methods of Surgical Data Science [ 5 ] that focus especially on the intraoperative setting with the aim to enable a personalized prediction of the surgical patient's outcome [ 10 ]. Surgomic features are characteristics of a surgical procedure which are automatically derived from multimodal intraoperative data to quantify processes in the operating room.…”

mentioning

confidence: 99%

“…Surgomic features are characteristics of a surgical procedure which are automatically derived from multimodal intraoperative data to quantify processes in the operating room. Based on the conceptual work of Wagner et al [ 10 ], we now aimed to develop surgomic features that are automatically derived from surgical videos by means of ML. The overarching aim of our work is thus to improve surgical therapy by extracting quantitative information from surgical data that may help to predict postoperative complications.…”

mentioning

confidence: 99%

Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study

Brandenburg,

Jenke,

Stern

et al. 2023

Surg Endosc

Self Cite

View full text Add to dashboard Cite

Background With Surgomics, we aim for personalized prediction of the patient's surgical outcome using machine-learning (ML) on multimodal intraoperative data to extract surgomic features as surgical process characteristics. As high-quality annotations by medical experts are crucial, but still a bottleneck, we prospectively investigate active learning (AL) to reduce annotation effort and present automatic recognition of surgomic features. Methods To establish a process for development of surgomic features, ten video-based features related to bleeding, as highly relevant intraoperative complication, were chosen. They comprise the amount of blood and smoke in the surgical field, six instruments, and two anatomic structures. Annotation of selected frames from robot-assisted minimally invasive esophagectomies was performed by at least three independent medical experts. To test whether AL reduces annotation effort, we performed a prospective annotation study comparing AL with equidistant sampling (EQS) for frame selection. Multiple Bayesian ResNet18 architectures were trained on a multicentric dataset, consisting of 22 videos from two centers. Results In total, 14,004 frames were tag annotated. A mean F1-score of 0.75 ± 0.16 was achieved for all features. The highest F1-score was achieved for the instruments (mean 0.80 ± 0.17). This result is also reflected in the inter-rater-agreement (1-rater-kappa > 0.82). Compared to EQS, AL showed better recognition results for the instruments with a significant difference in the McNemar test comparing correctness of predictions. Moreover, in contrast to EQS, AL selected more frames of the four less common instruments (1512 vs. 607 frames) and achieved higher F1-scores for common instruments while requiring less training frames. Conclusion We presented ten surgomic features relevant for bleeding events in esophageal surgery automatically extracted from surgical video using ML. AL showed the potential to reduce annotation effort while keeping ML performance high for selected features. The source code and the trained models are published open source. Graphical abstract

show abstract

Surgomics: personalized prediction of morbidity, mortality and long-term outcome in surgery using machine learning on multimodal data

Cited by 14 publications

References 66 publications

Prediction of Pathologic Complete Response for Rectal Cancer Based on Pre-treatment Factors Using Machine Learning

Prediction of Pathologic Complete Response for Rectal Cancer Based on Pre-treatment Factors Using Machine Learning

Artificial intelligence in pancreatic surgery: current applications

Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study

Contact Info

Product

Resources

About