Introduction
A limitation to expanding laparoscopic simulation training programs is the scarcity of expert evaluators. In 2019, a new digital platform for remote and asynchronous laparoscopic simulation training was validated. Through this platform, 369 trainees have been trained in 14 institutions across Latin America, collecting 6729 videos of laparoscopic training exercises. The use of artificial intelligence (AI) has recently emerged in surgical simulation, showing usefulness in training assessment, virtual reality scenarios, and laparoscopic virtual reality simulation. An AI algorithm to assess basic laparoscopic simulation training exercises was developed. This study aimed to analyze the agreement between this AI algorithm and expert evaluators in assessing basic laparoscopic-simulated training exercises.
Methods
The AI algorithm was trained using 400-bean drop (BD) and 480-peg transfer (PT) videos and tested using 64-BD and 43-PT randomly selected videos, not previously used to train the algorithm. The agreement between AI and expert evaluators from the digital platform (EE) was then analyzed. The exercises being assessed involve using laparoscopic graspers to move objects across an acrylic board without dropping any objects in a determined time (BD < 24 s, PT < 55 s). The AI algorithm can detect object movement, identify if objects have fallen, track grasper clamps location, and measure exercise time. Cohen’s Kappa test was used to evaluate the agreement between AI assessments and those performed by EE, using a pass/fail nomenclature based on the time to complete the exercise.
Results
After the algorithm was trained, 79.69% and 93.02% agreement were observed in BD and PT, respectively. The Kappa coefficients test observed for BD and PT were 0.59 (moderate agreement) and 0.86 (almost perfect agreement), respectively.
Conclusion
This first approach of AI use in basic laparoscopic skills simulated training assessment shows promising results, providing a preliminary framework to expand the use of AI to other basic laparoscopic skills exercises.
Supplementary Information
The online version contains supplementary material available at 10.1007/s00464-022-09576-1.
Background Worldwide, trauma-related deaths are one of the main causes of mortality. Appropriate surgical treatment is crucial to prevent mortality, however, in the past decade, general surgery residents' exposure to trauma cases has decreased, particularly since the COVID-19 pandemic. In this context, accessible simulation-based training scenarios are essential. Methods A low-cost, previously tested OSCE scenario for the evaluation of surgical skills in trauma was implemented as part of a short training boot camp for residents and recently graduated surgeons. The following stations were included bowel anastomosis, vascular anastomosis, penetrating lung injury, penetrating cardiac injury, and gastric perforation (laparoscopic suturing). A total of 75 participants from 15 different programs were recruited. Each station was videotaped in high definition and assessed in a remote and asynchronous manner. The level of competency was assessed through global and specific rating scales alongside procedural times. Self-confidence to perform the procedure as the leading surgeon was evaluated before and after training. Results Statistically significant differences were found in pre-training scores between groups for all stations. The lowest scores were obtained in the cardiac and lung injury stations. After training, participants significantly increased their level of competence in both grading systems. Procedural times for the pulmonary tractotomy, bowel anastomosis, and vascular anastomosis stations increased after training. A significant improvement in self-confidence was shown in all stations. Conclusion An OSCE scenario for training surgical skills in trauma was effective in improving proficiency level and selfconfidence. Low pre-training scores and level of confidence in the cardiac and lung injury stations represent a deficit in residency programs that should be addressed. The incorporation of simulation-based teaching tools at early stages in residency would be beneficial when future surgeons face extremely severe trauma scenarios.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.