Effect of head-mounted displays on students’ acquisition of surgical suturing techniques compared to an e-learning and tutor-led course: a randomized controlled trial

Abstract

Background: Although surgical suturing is one of the most important basic skills, many medical school graduates do not acquire sufficient knowledge of it due to its lack of integration into the curriculum or a shortage of tutors. E-learning approaches attempt to address this issue but still rely on the involvement of tutors. Furthermore, the learning experience and visual-spatial ability appear to play a critical role in surgical skill acquisition. Virtual reality head-mounted displays (HMDs) could address this, but the benefits of immersive and stereoscopic learning of surgical suturing techniques are still unclear. Material and methods: In this multi-arm randomized controlled trial, 150 novices participated. Three teaching modalities were compared: an e-learning course (monoscopic), an HMD-based course (stereoscopic, immersive), both self-directed and a tutor-led course with feedback. Suturing performance was recorded by video camera both before and after course participation (>26 h of video material) and assessed in a blinded fashion using the Objective Structured Assessment of Technical Skills (OSATS) Global Rating Score (GRS). Furthermore, the optical flow of the videos was determined using an algorithm. The number of sutures performed was counted, the visual-spatial ability was measured with the Mental Rotation Test (MRT), and courses were assessed with questionnaires. Results: Students’ self-assessment in the HMD-based course was comparable to that of the tutor-led course and significantly better than in the e-learning course (P=0.003). Course suitability was rated best for the tutor-led course (x̄=4.8), followed by the HMD-based (x̄=3.6) and e-learning (x̄=2.5) courses. The median ΔGRS between courses was comparable (P=0.15) at 12.4 (95% CI 10.0–12.7) for the e-learning course, 14.1 (95% CI 13.0–15.0) for the HMD-based course, and 12.7 (95% CI 10.3–14.2) for the tutor-led course. However, the ΔGRS was significantly correlated with the number of sutures performed during the training session (P=0.002), but not with visual-spatial ability (P=0.615). Optical flow (R 2=0.15, P<0.001) and the number of sutures performed (R 2=0.73, P<0.001) can be used as additional measures to GRS. Conclusion: The use of HMDs with stereoscopic and immersive video provides advantages in the learning experience and should be preferred over a traditional web application for e-learning. Contrary to expectations, feedback is not necessary for novices to achieve a sufficient level in suturing; only the number of surgical sutures performed during training is a good determinant of competence improvement. Nevertheless, feedback still enhances the learning experience. Therefore, automated assessment as an alternative feedback approach could further improve self-directed learning modalities. As a next step, the data from this study could be used to develop such automated AI-based assessments.

AIxSuture: vision-based assessment of open suturing skills

“…We evaluate two different methods for skill assessment on the AIxSuture data [ 19 ]. The first is a state of the art model developed by Funke et al [ 11 ] using an I3D model in combination with a temporal segment network (TSN) [ 12 ].…”

“…The AIxSuture dataset is the result of a study we recently published [ 19 , 20 ]. It is comprised of 314 videos of approximately 5 min each taken at 30 fps resulting in about 100 GB of video material.…”

“…The sum of all categories forms a global rating score (GRS), ranging from scores of 8 to 40. An initial analysis of the inter-rater reliability of three raters yielded an average pairwise Pearson correlation coefficient of [ 20 ].…”

“…Furthermore, a video dataset with labeled skill ratings has not yet been benchmarked for open surgical techniques. In this paper, we present a large dataset of open suturing recordings that we released in a recent publication [ 19 ]. We provide an in-depth analysis of the data as well as benchmarks for automatic skill assessment, including the first end-to-end trained approach for skill assessment for open surgery training.…”