At-home medical student simulation: achieving knot-tying proficiency using video-based assessment

Nagaraj, Madhuri B.; Campbell, Krystle; Rege, Robert V.; Mihalic, Angela P.; Scott, Daniel J.

doi:10.1007/s44186-022-00007-2

Cited by 3 publications

(11 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Students recognized the at-home training environment as a low-stress and convenient practice environment. Additionally, examination of that curriculum demonstrated reduced personnel needs associated with the virtual platform [11]. However, the need for more frequent and directed feedback was also reported.…”

mentioning

confidence: 99%

“…Video-based training has demonstrated many benefits, including relieving and Other Interventional Techniques Presented at Society of American Gastrointestinal and Endoscopic Surgeons Next Big Thing Poster Presentation. August 31-September 3, 2021. some of the personnel burden and scheduling issues, and allowing for retrospective review while maintaining equivalent learning outcomes [9][10][11]. There remains, however, limitations in the ability to provide directed, immediate feedback [10].…”

mentioning

confidence: 99%

“…We previous reported positive learner outcomes associated with an open suturing and knot-tying curriculum for our second-year medical students which we adapted to an at-home video-based assessment format due to the social distancing requirements of COVID-19 [11]. Students recognized the at-home training environment as a low-stress and convenient practice environment.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background Early introduction and distributed learning have been shown to improve student comfort with basic requisite suturing skills. The need for more frequent and directed feedback, however, remains an enduring concern for both remote and in-person training. A previous in-person curriculum for our second-year medical students transitioning to clerkships was adapted to an at-home video-based assessment model due to the social distancing implications of COVID-19. We aimed to develop an Artificial Intelligence (AI) model to perform video-based assessment. Methods Second-year medical students were asked to submit a video of a simple interrupted knot on a penrose drain with instrument tying technique after self-training to proficiency. Proficiency was defined as performing the task under two minutes with no critical errors. All the videos were first manually rated with a pass-fail rating and then subsequently underwent task segmentation. We developed and trained two AI models based on convolutional neural networks to identify errors (instrument holding and knot-tying) and provide automated ratings. Results A total of 229 medical student videos were reviewed (150 pass, 79 fail). Of those who failed, the critical error distribution was 15 knot-tying, 47 instrument-holding, and 17 multiple. A total of 216 videos were used to train the models after excluding the low-quality videos. A k-fold cross-validation (k = 10) was used. The accuracy of the instrument holding model was 89% with an F-1 score of 74%. For the knot-tying model, the accuracy was 91% with an F-1 score of 54%. Conclusions Medical students require assessment and directed feedback to better acquire surgical skill, but this is often time-consuming and inadequately done. AI techniques can instead be employed to perform automated surgical video analysis. Future work will optimize the current model to identify discrete errors in order to supplement video-based rating with specific feedback.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although the implementation of video-based assessments and digital feedback has been going on for a few years, its implementation rose exponentially between 2020 and 2021 due to the COVID-19 pandemic [ 13 , 14 , 22 – 24 ] . Among these, we found several studies that assessed the effectiveness of remote or distance-based simulation training for basic suturing skills with promising results [ 25 – 28 ]. Even though these studies did answer to the circumstances of the pandemic and allowed to maintain training, there is still a need for trainers to be synchronously present during the tutoring sessions to provide feedback.…”

Section: Discussionmentioning

confidence: 99%

“…Unlike previously published studies, the digital platform and teaching methodology implemented in this study allowed time flexibility for both trainers and trainees. First, it allowed trainers to assess videos and provide individualized feedback at any time and from anywhere in the world, without the need to find available time away from clinical labor to teach the trainee [ 25 – 27 , 29 , 30 ]. Second, it provided the trainee the chance to train from any place they want and as many times as they feel comfortable while having unlimited access to the video-enhanced feedback received.…”

Section: Discussionmentioning

confidence: 99%

Taking advantage of asynchronous digital feedback: development of an at-home basic suture skills training program for undergraduate medical students that facilitates skills retention

et al. 2023

View full text Add to dashboard Cite

Developing and Evaluating a Novel Vessel Ligation Simulator Designed for Teaching Surgical Knot-Tying: A Mixed-Methods Study

Wu,

Yau,

Lin

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: Surgical knot-tying is a fundamental learning subject for surgical novices. However, no existing knot-tying training simulator offers a direct objective measurement of knot security, a critical feature of professional performance in this skill. This study aims to bridge this gap by developing a novel knot-tying simulator capable of assessing knot security objectively. Its educational impact was also investigated through professional user evaluation.Methods: A three-phase mixed-methods research study was conducted from August 2021 to September 2022. The first phase involved developing and manufacturing a novel vessel ligation simulator that measured the pressure-holding ability of knots, serving as an indicator of knot security, through post-ligation pressure testing. The second phase involved the participation of expert surgeons (n = 6) and Year-5 medical students (n = 30) in product evaluation. The final phase was a mixed-methods data analysis to determine the educational impact of the simulator. The collected data included participants’ demographics, prior experience in surgical knot-tying, performance metrics on the simulator, experts’ responses to a five-point Likert scale survey of simulator validity, novices’ responses to a 10-point scale survey of self-confidence in mastering surgical knot-tying skills before and after the training course, and their overall perceptions regarding the learning experience associated with the simulator.Results: The simulator was crafted using 3D-printing technology. The experts gave it high scores in face and content validity, with mean scores of 4.5 and 4.77, respectively. Twenty-four novice participants (80%) achieved knot security in vessel ligation by the end of the 3-hour training course, though they required more time for task completion compared to experts. Furthermore, all novice participants reported a marked increase in their post-class self-confidence in skill mastery. The unanimous approval from the novices highlighted the simulator's effectiveness in enhancing their learning, especially its high-fidelity design and objective feedback. This facilitated their reflection and motivation for the post-class deliberate practice.Conclusion: This study not only validates our hypothesis regarding the appropriateness of pressure-holding ability in teaching surgical knot-tying but also establishes a comprehensive methodological framework beneficial for curriculum developers to design effective simulation-based skill training in surgery.

show abstract

At-home medical student simulation: achieving knot-tying proficiency using video-based assessment

Cited by 3 publications

References 17 publications

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching

Taking advantage of asynchronous digital feedback: development of an at-home basic suture skills training program for undergraduate medical students that facilitates skills retention

Developing and Evaluating a Novel Vessel Ligation Simulator Designed for Teaching Surgical Knot-Tying: A Mixed-Methods Study

Contact Info

Product

Resources

About