BACKGROUND Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. METHODS Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between self- and trainer-assessments was measured using weighted Cohen’s kappa coefficients. RESULTS Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5–28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between self- and trainer-assessments improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were greater than 0.81 after 18 to 21 task attempts. CONCLUSIONS Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.

No abstract

Objective Many bench-top surgical simulators assess laparoscopic proficiency, yet few address core open surgical skills. The purpose of this study is to describe a cost-effective bench-top vessel-ligation simulator and provide construct validation. Design Prospective comparison of blinded proficiency assessments among participants performing a bench-top vessel-ligation simulation task. Evaluations were performed using Objective Structured Assessments of Technical Skills. Setting This study took place at the University of Virginia School of Medicine, a large academic medical institution. Participants Participants included fourth-year medical students participating in a focused surgical elective course (N = 16), post-graduate year (PGY) 2–3 surgery residents (N = 6), and surgical faculty (N = 5). Results Fixed costs of the vessel-ligation simulator totaled $30. Flexible costs of operation were less than $0.20 per attempt. Median task-specific checklist scores among medical students, residents, and faculty were 4.83, 7.33, and 7.67, respectively. Median global rating scores across the three groups were 2.29, 4.43, and 4.76, respectively. Significant proficiency differences were noted between students and residents/faculty for both metrics (p < 0.001). Conclusions A cost-effective bench-top simulator can effectively measure proficiency with basic open surgical techniques such as vessel-ligation. Among junior surgical trainees, this tool can identify learning gaps and improve operative skills in a pre-clinical setting.

No abstract

Background A procedural training protocol for medical students must be cognizant of faculty opportunity costs, which may preclude individually-supervised practice. Meanwhile, sporadic exposure in large group settings yields suboptimal proficiency. The purpose of this study is to assess the effectiveness of undergraduate assistant instructors in providing one-on-one simulation-based instruction for basic invasive techniques. Materials and Methods Investigators designed proficiency-based checklists by faculty consensus for three simulation tasks: orotracheal intubation, central venous catheterization (CVC), and suturing. Four undergraduate students were trained as instructors. Inter-rater agreement between instructors using the task checklists ranged from 0.754 to 0.866. Instructors conducted up to seven flexibly-scheduled, one-on-one practice sessions with clinically inexperienced medical student participants. Sessions were comprised of repetitive task attempts with an evaluation after every attempt. Upon completion of the training protocol, participants underwent evaluation by an experienced surgeon blinded to practice session performance. Study participants were surveyed to assess satisfaction. Results Twenty-nine participants completed the study. Median total practice time was 8.75 hours (IQR 7.12-8.75). Post-test pass rates were 93% (26/28), 71% (20/28), and 68% (19/28) for suturing, intubation and CVC, respectively. Ninety-seven percent (27/28) of participants were satisfied with their experience and 62% (18/29) advocated for protocol adoption into the standard pre-clinical curriculum. Estimated cost saved by using student instructors, based on departmental collections for surgical faculty, was $43,760. Conclusions Clinically-inexperienced assistants may be trained as instructors for basic simulation tasks with excellent inter-rater reliability. Deploying these assistant instructors makes effective, one-on-one technical training for pre-clinical medical students financially feasible.