Measurement of Nontechnical Skills During Robotic-Assisted Surgery Using Sensor-Based Communication and Proximity Metrics

Abstract

Nontechnical skills (NTS) include cognitive and interpersonal (eg, teamwork) skills 1 ; they are linked to patient outcomes in surgery. 2,3 Currently, NTS assessment in surgery relies on behavior ratings that are subjective, potentially biasing assessments, and requires trained raters, a limited resource.This study demonstrates a sensing-based system for objectively measuring intraoperative NTS. We hypothesized that objective, sensor-derived communication and proximity metrics are associated with NTS scores assessed with existing tools and can accurately predict exemplar NTS scores that promote patient safety and teamwork. MethodsThis cohort study, conducted between January 1 and December 31, 2019, was approved by the Indiana University institutional review board and followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for observational studies. No race or ethnicity data were collected because those details might identify participants. The Non-Technical Skills for Surgeons (NOTSS) tool with reported validity evidence was used as the gold standard NTS assessment. Two raters assessed surgeon NTS, and interrater reliability was determined. 4 Three categories of objective metrics were captured with wearable sensors and video (Table ): communication, speech (Praat version 6.1.01), and proximity of surgical team member location (Figure).Surgical teams performing robotic-assisted surgeries (RAS) from general, colorectal, and urology specialties were recruited. After surgeons provided written consent, sensors and video were captured during RAS. Data were segmented into 5 phases: patient-side cart docking, 5 minutes before the critical phase, critical phase, 5 minutes after the critical phase, and 10 minutes before patient-side cart undocking.After multicollinearity was checked, metrics with significant association (2-sided hypothesis tests, α = .05) with NOTSS scores were identified with Pearson correlations (hypothesis 1) in RStudio version Rv1.3.1093 (RStudio, PBC) (Hmisc package). Two-sided P < .05 indicated statistical significance. Predictors of exemplar NTS behavior (highest NOTSS score; hypothesis 2) were modeled with linear and nonlinear machine-learning algorithms (validated with 3-fold cross validation; details in the eAppendix in the Supplement). ResultsThirty-four cases were observed, with the participation of 16 nurses (6 men [38%]; 10 women [63%]), 12 residents or fellows (6 men [50%]; 6 women [50%]), 11 anesthesiologists (7 men [64%]; 4 women [36%]), and 4 surgeons (4 men [100%]). Surgeons had a mean (SD) 15 (16) years of experience and performed RAS at least 4 hours per week. Surgeons' mean (SD) overall NOTSS score was 3.45 (0.43) (1 = poor, 4 = exemplar 4 ). The interrater reliability for the NOTSS score was moderate (0.58, based on 25 cases).

Evaluating Nontechnical Skills and Leadership Skills During Simulated Critical Care Scenarios

“…Participant NTS was evaluated using three metrics: Non- Technical Skills for Surgeons (NOTSS) score, leadership score (subject-matter evaluation on if necessary and appropriate leadership skill were displayed), and communication-based metrics. The communication metrics were used to decompose speech into key elements further discussed in Cha et al 2021.…”

“…Measuring NTS to improve patient safety has traditionally been done through trained raters’ behavior-based rating tools. In general, NTS assessment tools are checklist-based, time- consuming, and require trained raters to complete (Cha et al, 2021; Yule et al, 2006). Although objective metrics utilizing sensor-bases measures of various NTS constructs were found in a recent scoping review, it was found that none exist to measure leadership skills (Cha & Yu, 2022).…”

Effects of non-technical skill-based surgical education for trainees on herniorrhaphy outcomes

“…Non-technical skills (NTS) have recently been developed to improve surgical team performance. They are currently being developed for high-quality surgical procedures and applied in robotic surgery [ 2 , 3 ]. NTS is defined as “cognitive, social, and personal resource skills that complement technical skills and contribute to safe and efficient task performance.” [ 4 ] NTS includes several concepts of cognitive, social, and personal skills, such as situational awareness, decision-making, communication and teamwork, and leadership.…”

“…Non-technical skills (NTS) have recently been developed to improve surgical team performance. They are currently being developed for high-quality surgical procedures and applied in robotic surgery [2,3]. NTS is defined as "cognitive, social, and personal resource skills that complement technical skills and contribute to safe and efficient task performance."…”