Semiautomated Motion Tracking for Objective Skills Assessment in Otologic Surgery: A Pilot Study

Abstract

Perioperative teaching and feedback of technical performance are essential during surgical training but are limited by competing demands on faculty time, resident work-hour restrictions, and desire for efficient operating room utilization. The increasing use of high-definition video microscopy and endoscopy in otolaryngology offers opportunities for trainees and faculty to evaluate performance outside the operating room but still requires faculty time. Our hypothesis is that automated motion tracking via video analysis offers a way forward to provide more consistent and objective feedback for surgical trainees. In this study, otolaryngology trainees at various levels were recorded performing a cortical mastoidectomy on cadaveric temporal bones using standard surgical instrumentation and high-definition video cameras coupled to an operating microscope. Videos were postprocessed to automatically track the tip of otologic dissection instruments. Data were analyzed for key metrics potentially applicable to the global rating scale used in the Accreditation Council for Graduate Medical Education’s Objective Structured Assessments of Technical Skills.

Video Analysis of Otologic Instrument Movement During Resident Mastoidectomies

“…It is not surprising that mean drill speed increases with resident experience level, as similar findings have been reported in the studies of simulated and cadaveric mastoidectomies (14,17). These increases could be attributed to the increased confidence or efficiency a surgical resident obtains as they become more experienced in performing a mastoidectomy (18).…”

“…Video motion tracking software has scarcely been used to measure objective characteristics of instrument movement during a cortical mastoidectomy. One previous study primarily limited to cadaveric mastoidectomies found that increased drill motion correlated with surgical experience level (17). Most other studies on mastoidectomy performance metrics utilizing cadaveric or virtual temporal bone assessment platforms have relied on assessment by observers (14).…”

Automatic Surgical Skill Assessment System Based on Concordance of Standardized Surgical Field Development Using Artificial Intelligence

“…Previous attempts at automatic surgical skill assessment were mainly performed by tracking hand or surgical tool motion; however, motion tracking mainly evaluates instrument handling or dexterity alone. Although instrument handling and dexterity are important elements for automatic surgical skill assessment, surgical skill assessment needs to be evaluated from various perspectives.…”

Subjective vs Computerized Assessment of Surgeon Skill Level During Mastoidectomy

“…4 Research in general surgery has shown that complex motions of laparoscopic and suturing tasks can be numerically described, [5][6][7] that measurements can distinguish novice from expert surgeons, 8,9 and that subsequent surgical motions can be predicted. 10 Research in otology shows promise but remains largely limited to simulation 11 or cadaveric settings, 12 with a need for intraoperative evaluation.…”

“…4 Research in general surgery has shown that complex motions of laparoscopic and suturing tasks can be numerically described, [5][6][7] that measurements can distinguish novice from expert surgeons, 8,9 and that subsequent surgical motions can be predicted. 10 Research in otology shows promise but remains largely limited to simulation 11 or cadaveric settings, 12 with a need for intraoperative evaluation. This pilot study aimed to develop methods for capturing and measuring surgical motion during intraoperative recordings of mastoidectomies and to compare the accuracy of objective metrics against subjective assessment in determining the experience level of the surgeon operating.…”