The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review

Balvardi, Saba; Kammili, Anitha; Hanson, Melissa N.; Mueller, Carmen L.; Vassiliou, Melina C.; Lee, Lawrence; Schwartzman, Kevin; Fiore, Julio F.; Feldman, Liane S.

doi:10.1007/s00464-022-09296-6

Cited by 22 publications

(13 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, very few articles met eligibility criteria, which limits the interpretation and indicates a significant opportunity for expanded, structured research. Nonetheless, our results are consistent with other studies that explore surgeon technical performance and clinical outcomes, in particular, a recently published systematic review by Balvardi et al 46 …”

Section: Discussionsupporting

confidence: 93%

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis

Woods¹,

Liberman

Rui

et al. 2023

JSLS

View full text Add to dashboard Cite

Section: Discussionsupporting

confidence: 93%

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis

Woods¹,

Liberman

Rui

et al. 2023

JSLS

View full text Add to dashboard Cite

“…Many researchers have conducted studies to determine preoperative and intraoperative parameters [17][18][19] that may be useful to identify patients at risk of PPUI. Recent studies have shown a correlation between good surgeon skill and excellent continence outcomes after RARP, [20][21][22] which has been proven by AI methods using semantic segmentation and automated performance status (APM). These findings strongly suggest that PPUI is influenced by surgeon (technical) and patient (anatomical) factors.…”

Section: Discussionmentioning

confidence: 98%

Combination of deep learning and ensemble machine learning using intraoperative video images strongly predicts recovery of urinary continence after robot‐assisted radical prostatectomy

et al. 2023

View full text Add to dashboard Cite

BackgroundWe recently reported the importance of deep learning (DL) of pelvic magnetic resonance imaging in predicting the degree of urinary incontinence (UI) following robot‐assisted radical prostatectomy (RARP). However, our results were limited because the prediction accuracy was approximately 70%.AimTo develop a more precise prediction model that can inform patients about UI recovery post‐RARP surgery using a DL model based on intraoperative video images.Methods and ResultsThe study cohort comprised of 101 patients with localized prostate cancer undergoing RARP. Three snapshots from intraoperative video recordings showing the pelvic cavity (prior to bladder neck incision, immediately following prostate removal, and after vesicourethral anastomosis) were evaluated, including pre‐ and intraoperative parameters. We evaluated the DL model plus simple or ensemble machine learning (ML), and the area under the receiver operating characteristic curve (AUC) was analyzed through sensitivity and specificity. Of 101, 64 and 37 patients demonstrated “early continence (using 0 or 1 safety pad at 3 months post‐RARP)” and “late continence (others),” respectively, at 3 months postoperatively. The combination of DL and simple ML using intraoperative video snapshots with clinicopathological parameters had a notably high performance (AUC, 0.683–0.749) to predict early recovery from UI after surgery. Furthermore, combining DL with ensemble artificial neural network using intraoperative video snapshots had the highest performance (AUC, 0.882; sensitivity, 92.2%; specificity, 78.4%; overall accuracy, 85.3%) to predict early recovery from post‐RARP incontinence, with similar results by internal validation. The addition of clinicopathological parameters showed no additive effects for each analysis using DL, EL and simple ML.ConclusionOur findings suggest that the DL algorithm with intraoperative video imaging is a reliable method for informing patients about the severity of their recovery from UI after RARP, although it is not clear if our methods are reproducible for predicting long‐term UI and pad‐free continence.

show abstract

“…18,19 Stulberg et al 20 used VBAs to rate the technical skills of 17 surgeons and demonstrated that differences in technical skill explained 26% of the variation in these surgeons' risk-adjusted complication rates following laparoscopic colectomy. A systematic review of VBAs across multiple surgical subspecialties also supported an association between improved technical performance on VBAs and more favorable shortterm postoperative outcomes such as unplanned reoperation or readmission, 21 suggesting that these types of assessment tools could be a useful option for surgical education researchers moving forward.…”

Section: Video-based Assessmentsmentioning

confidence: 94%

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

Montgomery

Lindeman

2023

Academic Medicine

View full text Add to dashboard Cite

In 2013, U.S. general surgery residency programs implemented a milestones assessment framework in an effort to incorporate more competency-focused evaluation methods. Developed by a group of surgical education leaders and other stakeholders working with the Accreditation Council for Graduate Medical Education and recently updated in a version 2.0, the surgery milestones framework is centered around 6 “core competencies”: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. While prior work has focused on the validity of milestones as a measure of resident performance, associations between general surgery resident milestone ratings and their post-training patient outcomes have only recently been explored in an analysis in this issue of Academic Medicine by Kendrick et al. Despite their well-designed efforts to tackle this complex problem, no relationships were identified. This accompanying commentary discusses the broader implications for the use of milestone ratings beyond their intended application, alternative assessment methods, and the challenges of developing predictive assessments in the complex setting of surgical care. Although milestone ratings have not been shown to provide the specificity needed to predict clinical outcomes in the complex settings studied by Kendrick et al, hope remains that utilization of other outcomes, assessment frameworks, and data analytic tools could augment these models and further our progress toward a predictive assessment in surgical education. Evaluation of residents in general surgery residency programs has grown both more sophisticated and complicated in the setting of increasing patient and case complexity, constraints on time, and regulation of resident supervision in the operating room. Over the last decade, surgical education research efforts related to resident assessment have focused on measuring performance through accurate and reproducible methods with evidence for their validity, as well as on attempting to refine decision making about resident preparedness for unsupervised practice.

show abstract

The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review

Cited by 22 publications

References 53 publications

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis

Association between Surgical Technical Skills and Clinical Outcomes: A Systematic Literature Review and Meta-Analysis

Combination of deep learning and ensemble machine learning using intraoperative video images strongly predicts recovery of urinary continence after robot‐assisted radical prostatectomy

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

Contact Info

Product

Resources

About