Construct validity of the Surgical Autonomy Program for the training of neurosurgical residents

Abstract

OBJECTIVE There is no standard way in which physicians teach or evaluate surgical residents intraoperatively, and residents are proving to not be fully competent at core surgical procedures upon graduating. The Surgical Autonomy Program (SAP) is a novel educational model that combines a modified version of the Zwisch scale with Vygotsky’s social learning theory. The objective of this study was to establish preliminary validity evidence that SAP is a reliable measure of autonomy and a useful tool for tracking competency over time. METHODS The SAP breaks each surgical case into 4 parts, or zones of proximal development (ZPDs). Residents are evaluated on a 4-tier autonomy scale (TAGS scale) for each ZPD in every surgical case. Attendings were provided with a teaching session about SAP and identified appropriate ZPDs for surgical cases under their area of expertise. All neurosurgery residents at Duke University Hospital from July 2017 to July 2021 participated in this study. Chi-square tests and ordinal logistic regression were used for the analyses. RESULTS Between 2017 and 2021, there were 4885 cases logged by 27 residents. There were 30 attendings who evaluated residents using SAP. Faculty completed evaluations on 91% of cases. The ZPD of focus directly correlated with year of residency (postgraduate year) (χ2 = 1221.1, df = 15, p < 0.001). The autonomy level increased with year of residency (χ2 = 3553.5, df = 15, p < 0.001). An ordinal regression analysis showed that for every year increase in postgraduate year, the odds of operating at a higher level of independence was 2.16 times greater (95% CI 2.11–2.21, p < 0.001). The odds of residents performing with greater autonomy was lowest for the most complex portion of the case (ZPD3) (OR 0.18, 95% CI 0.17–0.20, p < 0.001). Residents have less autonomy with increased case complexity (χ2 = 160.28, df = 6, p < 0.001). Compared with average cases, residents were more likely to operate with greater autonomy on easy cases (OR 1.44, 95% CI 1.29–1.61, p < 0.001) and less likely to do so on difficult cases (OR 0.72, 95% CI 0.67–0.77, p < 0.001). CONCLUSIONS This study demonstrates preliminary evidence supporting the construct validity of the SAP. This tool successfully tracks resident autonomy and progress over time. The authors’ smartphone application was widely used among surgical faculty and residents, supporting integration into the perioperative workflow. Wide implementation of SAP across multiple surgical centers will aid in the movement toward a competency-based residency education system.

A Scoping Review of Professionalism in Neurosurgery

“…Articles examining professional development ranged from resident training to maintenance of certification. 118-151 The most common type of publication in this category was preintervention and postintervention studies. 123,124,128,130,131,136,147,148 The development and implementation of the ACGME milestones was discussed in several articles.…”

“…118-151 The most common type of publication in this category was preintervention and postintervention studies. 123,124,128,130,131,136,147,148 The development and implementation of the ACGME milestones was discussed in several articles. 129,134,145,150 Murad et al 123 discussed the implementation of a transition to practice program.…”

Advances and Challenges in Minimally Invasive Spine Surgery

“…Within the sphere of minimally invasive surgery, a tubular microdiscectomy would be segmented into four parts (i.e., Part 1: tube placement and docking, Part 2: laminectomy, Part 3: removal of ligamentum and dural exposure, Part 4: discectomy), which are each then further segmented into varying grades of independence (i.e., Zone 1: observation, Zone 2: performance with direct supervision, Zone 3: performance with minimal supervision, Zone 4: operator independence). Collection of data across time has allowed faculty members to better gauge the skill level of their trainees, and helps trainees understand their areas of strength and weakness [ 78 ].…”

Expertise in surgical neuro-oncology. Results of a survey by the EANS neuro-oncology section

“…The basic idea that experience translates into expertise seems often confirmed in everyday neurosurgical practice, including brain tumor surgery. Neurosurgery relies heavily on technical skills, and it is not surprising that quite a few researchers have investigated how technical competency can be assessed and taught ( Gelinas-Phaneuf and Del Maestro, 2013 ; Kirsch et al, 2022 ; Davids et al, 2021 ). More recently, some groups have investigated a potential role for modern computer-based techniques such as virtual reality simulation and machine learning or virtual communities in this field ( Winkler-Schwartz et al, 2019 ; Titov et al, 2023 ; Gandamihardja, 2014 ; Bonrath et al, 2015 ; Issenberg et al, 2005 ; Ledwos et al, 2022 ).…”