Do Resident Surgical Volumes and Level of Training Correlate with Improved Performance on Psychomotor Skills Tasks: Construct Validity Testing of an ASSH Training Platform (STEP)?

Abstract

Background: The Surgical Training and Educational Platform (STEP) was developed by the American Society for Surgery of the Hand (ASSH) as a cost-effective set of surgical simulation modules designed to assess critical skills in hand surgery. Previous study demonstrated that STEP can differentiate between novice trainees and board-certified, certificate of added qualification hand surgeons. The purpose of this study was to assess construct validity of STEP by testing its ability to differentiate psychomotor skill level among intermediate trainees. Methods: We evaluated 30 residents from 2 orthopaedic residency programs on 8 modules: (1) lag screw fixation, (2) depth of plunge during bicortical drilling, (3) flexor tendon repair, (4) phalangeal fracture pinning, (5) central axis scaphoid fixation, (6) full-thickness skin graft harvest, (7) microsurgery, and (8) wrist arthroscopy. Spearman correlation was used to correlate total and task-specific scores to case log numbers, months in training, and number of hand surgery rotations. Results: Senior residents had significantly higher mean number of total cases in their total case log (mean difference 96.2, 95% confidence interval [CI] 67.5-124.8, p < 0.01) and number of task-specific cases. Moderate correlation was observed between case log numbers and scaphoid fixation score (rs = 0.423, 95% CI 0.07-0.69) and total score (rs = 0.584, 95% CI 0.25-0.79). Moderate correlation was observed between months in training with: scaphoid fixation (rs = 0.377, 95% CI 0.01-0.66) and microsurgery (rs = 0.483, 95% CI 0.13-0.73); strong correlation was seen with total score (rs = 0.656, 95% CI 0.35-0.83). Moderate correlation with number of hand surgery rotations was observed with tendon repair (rs = 0.362, 95% CI −0.01 to 0.65), skin graft (rs = 0.385, 95% CI 0.01-0.66), wrist arthroscopy (rs = 0.391, 95% CI 0.02-0.67), microsurgery (rs = 0.461, 95% CI 0.10-0.71), and scaphoid fixation (rs = 0.578, 95% CI 0.25-0.79); and strong correlation was seen with total score (rs = 0.670, 95% CI 0.37-0.84). Discussion/Conclusion: The STEP is a validated ASSH education tool that provides a cost-effective simulation for the assessment of fundamental psychomotor skills in hand surgery. Total STEP score correlated with total task-related case volumes as well as months in training and number of hand rotations. Scoring could be modified to improve the fidelity of assessing surgical performance. And, while both time and human resource consuming to perform, administer, and score, this study demonstrates construct validity of STEP in assessing the progression of surgical skill through residency.

Impact of autonomic regulation on burnout and performance in thoracic surgery residents

“…Within surgical training, workload, as quantified by operative volume, correlates with increased performance, as measured by ACGME milestones and simulated procedural tasks. 29 , 30 , 31 , 32 With workload leading to both increased performance and emotional exhaustion, it is something of a double-edged sword for surgical training programs. HRV may offer a mechanism by which we can allow the resident to push to the limits of their capacity and safely monitor for adequate recovery allowing for maximum efficiency and improved performance.…”

Orthopaedic Hand Surgical Simulation Training: A Review

“…Such tools have been validated, with performance on the simulators correlating with level of training, case volume, and number of hand rotations completed. 7 Dental cotton rolls or bungee cords are used to practice tendon repair 8 9 ( Fig. 1B ).…”

On-Call Workload Differences in Neurosurgery: Resident Call “Weather” and a Departure from Colored Cloud Labels

“…1,3 The impact of any potential differences could be amplified in procedurally oriented specialties, where skill acquisition is linked to episodic practice and volume of experience. [5][6][7][8][9][10] A junior neurosurgery resident's on-call workload depends largely on the number of new patient evaluations, bedside procedures, and operative emergencies. 11 According to timeworn terminology, residents who admit few patients, all with straightforward problems requiring few procedures, are referred to as "white clouds."…”

“…1,3 The impact of any potential differences could be amplified in procedurally oriented specialties, where skill acquisition is linked to episodic practice and volume of experience. 5-10…”