3D Autonomous Surgeon’s Hand Movement Assessment Using a Cascaded Fuzzy Supervisor in Multi-Thread Video Processing

Abstract

The purpose of the Fundamentals of Laparoscopic Surgery (FLS) training is to develop laparoscopic surgery skills by using simulation experiences. Several advanced training methods based on simulation have been created to enable training in a non-patient environment. Laparoscopic box trainers—cheap, portable devices—have been deployed for a while to offer training opportunities, competence evaluations, and performance reviews. However, the trainees must be under the supervision of medical experts who can evaluate their abilities, which is an expensive and time-consuming operation. Thus, a high level of surgical skill, determined by assessment, is necessary to prevent any intraoperative issues and malfunctions during a real laparoscopic procedure and during human intervention. To guarantee that the use of laparoscopic surgical training methods results in surgical skill improvement, it is necessary to measure and assess surgeons’ skills during tests. We used our intelligent box-trainer system (IBTS) as a platform for skill training. The main aim of this study was to monitor the surgeon’s hands’ movement within a predefined field of interest. To evaluate the surgeons’ hands’ movement in 3D space, an autonomous evaluation system using two cameras and multi-thread video processing is proposed. This method works by detecting laparoscopic instruments and using a cascaded fuzzy logic assessment system. It is composed of two fuzzy logic systems executing in parallel. The first level assesses the left and right-hand movements simultaneously. Its outputs are cascaded by the final fuzzy logic assessment at the second level. This algorithm is completely autonomous and removes the need for any human monitoring or intervention. The experimental work included nine physicians (surgeons and residents) from the surgery and obstetrics/gynecology (OB/GYN) residency programs at WMU Homer Stryker MD School of Medicine (WMed) with different levels of laparoscopic skills and experience. They were recruited to participate in the peg-transfer task. The participants’ performances were assessed, and the videos were recorded throughout the exercises. The results were delivered autonomously about 10 s after the experiments were concluded. In the future, we plan to increase the computing power of the IBTS to achieve real-time performance assessment.

Advances in gynecologic simulation: implementation, validity, and new resources

“…They used two cameras, a front and top camera, to create a 3D field which increased object detection. Ultimately, they were successful in creating a reliable artificial intelligence algorithm to assess the peg transfer task of both prerecorded videos and live videos (with 10 s delay), but it was not compared with a human evaluator [20]. Both studies demonstrate not only the potential for artificial intelligence technology but also the progress that needs to be made before this can replicate the benefits of a live expert surgeon educator.…”

Complex thinking through a Transition Design-guided Ideathon: testing an AI platform on the topic of sharing economy

“…Forty-seven documents were identified. Documents related to the health area (Lamberti et al, 2019;McNamara et al, 2019;Shorey et al, 2019;Rajadhyaksha, 2020;Zhao et al, 2020;Cheng et al, 2022;Creed et al, 2022;Liaw et al, 2022;Lokala et al, 2022;Shah et al, 2023), industry (Barykin et al, 2020;Mokhtarname et al, 2020;Sandner et al, 2020;Dmitrievsky et al, 2022;Obermayer et al, 2022;Zakharkina et al, 2022), education (Hrich et al, 2019;Tsalapatas et al, 2019;Cortés et al, 2020;Paba-Medina et al, 2020;Raj et al, 2020;Yang et al, 2020;Demchenko et al, 2021;Hurajová, 2021;Jiang, 2021;Petrescu et al, 2021Ghnemat et al, 2022Polak et al, 2022;Ramírez-Montoya et al, 2022;Rataj and Wojcik, 2022), science (Bruneckiene et al, 2019;Desnos et al, 2022;Ramírez-Montoya et al, 2022;Zhu et al, 2022), evaluation (Kiran et al, 2019;Prom et al, 2019;Konys, 2020;Bachiri and Mouncif, 2023;Rashidi Fathabadi et al, 2023), engineering (Kaspar and Vielhaber, 2019;Telnov and Korovin, 2019)...…”

Towards Real-Time Multi-Class Object Detection and Tracking for the FLS Pattern Cutting Task

“…However, the system had limitations when the instruments or objects were not within view of the camera. To address this issue, a new autonomous evaluation system was proposed by authors in [22], which utilized two cameras and multi-thread video processing to detect laparoscopic instruments.…”