What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature

Abstract

A crucial element of any surgical training program is the ability to provide procedure-specific, objective, and reliable measures of performance. During robotic surgery, objective clinically relevant performance metrics (CRPMs) can provide tailored contextual feedback and correlate with clinical outcomes. This review aims to define CRPMs, assess their validity in robotic surgical training and compare CRPMs to existing measures of robotic performance. A systematic search of Medline and Embase databases was conducted in May 2022 following the PRISMA guidelines. The search terms included Clinically Relevant Performance Metrics (CRPMs) OR Clinically Relevant Outcome Measures (CROMs) AND robotic surgery. The study settings, speciality, operative context, study design, metric details, and validation status were extracted and analysed. The initial search yielded 116 citations, of which 6 were included. Citation searching identified 3 additional studies, resulting in 9 studies included in this review. Metrics were defined as CRPMs, CROMs, proficiency-based performance metrics and reference-procedure metrics which were developed using a modified Delphi methodology. All metrics underwent both contents and construct validation. Two studies found a strong correlation with GEARS but none correlated their metrics with patient outcome data. CRPMs are a validated and objective approach for assessing trainee proficiency. Evaluating CRPMs with other robotic-assessment tools will facilitate a multimodal metric evaluation approach to robotic surgery training. Further studies should assess the correlation with clinical outcomes. This review highlights there is significant scope for the development and validation of CRPMs to establish proficiency-based progression curricula that can be translated from a simulation setting into clinical practice.

Predicting Surgical Experience After Robotic Nerve-sparing Radical Prostatectomy Simulation Using a Machine Learning–based Multimodal Analysis of Objective Performance Metrics

“…Similar studies used ML to identify surgical gestures from video review and the differences in patterns in surgeons with different skill levels. 14 However, there are several major distinctions between our study and previous efforts. Our work was performed using a standardized simulation platform—removing the need to control for patient variables.…”

“…13 OPIs were validated with video analysis using GEARS during nerve-sparing RARP, highlighting skill differences between various expert surgeons while correlating with clinical outcomes. 14 Deeper into the hierarchical analysis of technical performance are surgical gestures, representing actions via instruments during a surgical step, which have been used to quantify suturing performance during vesicoureteral anastomosis and in predicting urinary continence recovery. 14 Simulation-based assessment provides a standardized and reproducible method to assess surgical skill by removing anatomical variables inherent with real patients.…”

“…14 Deeper into the hierarchical analysis of technical performance are surgical gestures, representing actions via instruments during a surgical step, which have been used to quantify suturing performance during vesicoureteral anastomosis and in predicting urinary continence recovery. 14 Simulation-based assessment provides a standardized and reproducible method to assess surgical skill by removing anatomical variables inherent with real patients. We have developed and validated a high-fidelity simulation model for nerve-sparing RARP, incorporating force sensors in the NVB to measure applied tissue tension during dissection.…”

“…14 Deeper into the hierarchical analysis of technical performance are surgical gestures, representing actions via instruments during a surgical step, which have been used to quantify suturing performance during vesicoureteral anastomosis and in predicting urinary continence recovery. 14…”

The Future of Surgical Skills Evaluation: What Is on Your Wish List?

“…5 Now that we have validated training tools and an expanding list of training targets, I think it is useful to pause and ask the question: what do we wish for next in robotic surgery training? Your answer is probably as good as mine, but here are my 3 top choices: (1) optimized knowledge on which aspects of surgical skills evaluations correlate with clinical outcomes, 6 (2) optimized curricula that prepare trainees for console performance, and (3) a pathway towards identifying and correcting underperforming trainees. Now let's comment further on the September issue of Urology Practice Ò , as we can up the stakes into the realms of artificial intelligence, machine learning, and deep learning.…”