Tailored Feedback Based on Clinically Relevant Performance Metrics Expedites the Acquisition of Robotic Suturing Skills—An Unblinded Pilot Randomized Controlled Trial

Abstract

Purpose:Previously, we identified 8 objective suturing performance metrics highly predictive of urinary continence recovery after robotic-assisted radical prostatectomy. Here, we aimed to test the feasibility of providing tailored feedback based upon these clinically relevant metrics and explore the impact on the acquisition of robotic suturing skills.Materials and Methods:Training surgeons were recruited and randomized to a feedback group or a control group. Both groups completed a baseline, midterm and final dry laboratory vesicourethral anastomosis (VUA) and underwent 4 intervening training sessions each, consisting of 3 suturing exercises. Eight performance metrics were recorded during each exercise: 4 automated performance metrics (derived from kinematic and system events data of the da Vinci® Robotic System) representing efficiency and console manipulation competency, and 4 suturing technical skill scores. The feedback group received tailored feedback (a visual diagram+verbal instructions+video examples) based on these metrics after each session. Generalized linear mixed model was used to compare metric improvement (Δ) from baseline to the midterm and final VUA.Results:Twenty-three participants were randomized to the feedback group (11) or the control group (12). Demographic data and baseline VUA metrics were comparable between groups. The feedback group showed greater improvement than the control group in aggregate suturing scores at midterm (mean Δ feedback group 4.5 vs Δ control group 1.1) and final VUA (Δ feedback group 5.3 vs Δ control group 4.9). The feedback group also showed greater improvement in the majority of the included metrics at midterm and final VUA.Conclusions:Tailored feedback based on specific, clinically relevant performance metrics is feasible and may expedite the acquisition of robotic suturing skills.

What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature

“…Together, their description of metrics included CRPMs, CROMs, PBP metrics, and reference-procedure metrics. Countries of publication included Germany [11,12], England [13][14][15][16][17], and the USA [6,18].…”

“…Ghazi et al, defined CROMs as measures that "extend beyond basic robotic skills training into procedure-specific training" and provide tailored feedback to allow surgeons to progress based on individualised capabilities [15]. Ma et al, stated that CRPMs were those utilized to provide procedure-tailored feedback for surgical training and therefore "expedite the acquisition of robotic suturing skills" for each individual surgeon [18]. Other terminology utilised in the included publications were "procedure specific assessment tools" that provided an objective assessment of robotic intraoperative performance and enabled tailored training feedback to achieve competency [6,12].…”

“…The metrics included: applied force to the neurovascular bundle during dissection, post-simulation margin status, UVA integrity, task-specific operating tasks, estimated blood loss [17] as well as console time, warm ischemia time (WIT), and positive surgical margins (PSMs) [15], respectively. Methodology for developing clinically relevant metrics for UVA utilised pre-existing validated metrics including APMs and RACE score [18]. The remaining 6 articles used a modified Delphi process, to identify and describe specific metrics for a reference procedure.…”

“…An iterative development process was used to assess feedback and the feasibility of the CROMs in relation to the RAPN [15]. APMs related to UVA steps were collated from data from the Da Vinci robotic system, and combined with technical skill scores from RACE, which was previously validated [18]. Considering the articles that utilised a Delphi panel to create their reference metrics, content validation was achieved by voting upon each metric, and ensuring high-level consensus was achieved before the metrics were accepted and included as part of the finalised reference metrics [6,[11][12][13][14]16].…”

This Month in Adult Urology

“…Ma et al (page 414) from Los Angeles, California and Norcross, Georgia, in a relatively uncommon educational randomized, controlled trial, use direct individualized feedback generated through automated and expert assessment of trainee technical skills to improve performance in a simulated environment. 3 The feedback was formally structured and tailored according to errors made in previous sessions. The results confirm that tailored feedback can be made efficient, and lead to measurable improvements in technical skills.…”