Surgical anatomy is taught early in medical school training. The literature shows that many physicians, especially surgical specialists, think that anatomical knowledge of medical students is inadequate and nesting of anatomical sciences later in the clinical curriculum may be necessary. Quantitative data concerning this perception of an anatomical knowledge deficit are lacking, as are specifics as to what content should be reinforced. This study identifies baseline areas of strength and weakness in the surgical anatomy knowledge of medical students entering surgical rotations. Third-year medical students completed a 20-25-question test at the beginning of the General Surgery and Obstetrics and Gynecology rotations. Knowledge of inguinal anatomy (45.3%), orientation in abdominal cavity (38.8%), colon (27.7%), and esophageal varices (12.8%) was poor. The numbers in parentheses are the percentage of questions answered correctly per topic. In comparing those scores to matched test items from this cohort as first-year students in the anatomy course, the drop in retention overall was very significant (P = 0.009) from 86.9 to 51.5%. Students also scored lower in questions relating to pelvic organs (46.7%), urogenital development (54.0%), pulmonary development (17.8%), and pregnancy (17.8%). These data showed that indeed, knowledge of surgical anatomy is poor for medical students entering surgical clerkships. These data collected will be utilized to create interactive learning modules, aimed at improving clinically relevant anatomical knowledge retention. These modules, which will be available to students during their inpatient surgical rotations, connect basic anatomy principles to clinical cases, with the ultimate goal of closing the anatomical knowledge gap.

No abstract

No abstract

Objective: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs. Summary of Background Data: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares. Methods: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis. Results: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001). Conclusions: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.

Purpose Learning is markedly improved with high-quality feedback, yet assuring the quality of feedback is difficult to achieve at scale. Natural language processing (NLP) algorithms may be useful in this context as they can automatically classify large volumes of narrative data. However, it is unknown if NLP models can accurately evaluate surgical trainee feedback. This study evaluated which NLP techniques best classify the quality of surgical trainee formative feedback recorded as part of a workplace assessment. Method During the 2016–2017 academic year, the SIMPL (Society for Improving Medical Professional Learning) app was used to record operative performance narrative feedback for residents at 3 university-based general surgery residency training programs. Feedback comments were collected for a sample of residents representing all 5 postgraduate year levels and coded for quality. In May 2019, the coded comments were then used to train NLP models to automatically classify the quality of feedback across 4 categories (effective, mediocre, ineffective, or other). Models included support vector machines (SVM), logistic regression, gradient boosted trees, naive Bayes, and random forests. The primary outcome was mean classification accuracy. Results The authors manually coded the quality of 600 recorded feedback comments. Those data were used to train NLP models to automatically classify the quality of feedback across 4 categories. The NLP model using an SVM algorithm yielded a maximum mean accuracy of 0.64 (standard deviation, 0.01). When the classification task was modified to distinguish only high-quality vs low-quality feedback, maximum mean accuracy was 0.83, again with SVM. Conclusions To the authors’ knowledge, this is the first study to examine the use of NLP for classifying feedback quality. SVM NLP models demonstrated the ability to automatically classify the quality of surgical trainee evaluations. Larger training datasets would likely further increase accuracy.