Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review

Abstract

Objective The purpose of this study was to identify, synthesize, and discuss objective behavioral or physiological metrics of surgeons’ nontechnical skills (NTS) in the literature. Background NTS, or interpersonal or cognitive skills, have been identified to contribute to safe and efficient surgical performance; however, current assessments are subjective, checklist-based tools. Intraoperative skill evaluation, such as technical skills, has been previously utilized as an objective measure to address such limitations. Methods Five databases in engineering, behavioral science, and medicine were searched following PRISMA reporting guidelines. Eligibility criteria included studies with NTS objective measurements, surgeons, and took place within simulated or live operations. Results Twenty-three articles were included in this review. Objective metrics included communication metrics and measures from physiological responses such as changes in brain activation and motion of the eye. Frequencies of content-coded communication in surgery were utilized in 16 studies and were associated with not only the communication construct but also cognitive constructs of situation awareness and decision making. This indicates the underlying importance of communication in evaluating the NTS constructs. To synthesize the scoped literature, a framework based on the one-way communication model was used to map the objective measures to NTS constructs. Conclusion Objective NTS measurement of surgeons is still preliminary, and future work on leveraging objective metrics in parallel with current assessment tools is needed. Application Findings from this work identify objective NTS metrics for measurement applications in a surgical environment.

Continuous Monitoring of Mental Load During Virtual Simulator Training for Laparoscopic Surgery Reflects Laparoscopic Dexterity: A Comparative Study Using a Novel Wireless Device

“…Due to the unobtrusive nature of mobile EEG devices, such devices can be used by surgeons during live operations. Intraoperative evaluation can provide an objective metric to assess surgeons’ performance in real-life scenarios and measure additional unobservable behaviors, such as mental readiness ( Cha and Yu, 2021 ).…”

Protocol for a scoping review on ‘surgical sabermetrics:’ technology-enhanced measurement of operative non-technical skills

“…technology, the operating theatre (OR), the surgery and the patient themselves. [10][11][12] Additionally, surgeons are humans, fallible and subject to influence from external and internal forces, including: the busy healthcare environment, fatigue, burnout, noise, stress at home and the need to focus on outpatients, inpatients and the current operation concurrently. NTSs are defined as the cognitive and social skills that characterise high performing individuals and teams.…”

“…As of writing, the authors are not aware of any current relevant reviews that focus on real-time, bias free, digital data. A recent scoping review by Cha and Yu 12 looked at objective measures of NTS, but did not specifically look at automated, digital measurements. Many studies included in this review required human expertise as part of the objective measurement.…”

“…There is a direct link between performance and clinical outcomes 9. Surgical performance is affected by several factors relating to interlinking technical and NTS, including the surgeon, the surgical team, equipment and technology, the operating theatre (OR), the surgery and the patient themselves 10–12. Additionally, surgeons are humans, fallible and subject to influence from external and internal forces, including: the busy healthcare environment, fatigue, burnout, noise, stress at home and the need to focus on outpatients, inpatients and the current operation concurrently.…”

Measurement of Nontechnical Skills During Robotic-Assisted Surgery Using Sensor-Based Communication and Proximity Metrics

“…Nontechnical skills (NTS) include cognitive and interpersonal (eg, teamwork) skills 1 ; they are linked to patient outcomes in surgery. 2,3 Currently, NTS assessment in surgery relies on behavior ratings that are subjective, potentially biasing assessments, and requires trained raters, a limited resource.…”