Artificial Intelligence-Based Hazard Detection in Robotic-Assisted Single-Incision Oncologic Surgery

Abstract

The problem: Single-incision surgery is a complex procedure in which any additional information automatically collected from the operating field can be of significance. While the use of robotic devices has greatly improved surgical outcomes, there are still many unresolved issues. One of the major surgical complications, with higher occurrence in cancer patients, is intraoperative hemorrhages, which if detected early, can be more efficiently controlled. Aim: This paper proposes a hazard detection system which incorporates the advantages of both Artificial Intelligence (AI) and Augmented Reality (AR) agents, capable of identifying, in real-time, intraoperative bleedings, which are subsequently displayed on a Hololens 2 device. Methods: The authors explored the different techniques for real-time processing and determined, based on a critical analysis, that YOLOv5 is one of the most promising solutions. An innovative, real-time, bleeding detection system, developed using the YOLOv5 algorithm and the Hololens 2 device, was evaluated on different surgical procedures and tested in multiple configurations to obtain the optimal prediction time and accuracy. Results: The detection system was able to identify the bleeding occurrence in multiple surgical procedures with a high rate of accuracy. Once detected, the area of interest was marked with a bounding box and displayed on the Hololens 2 device. During the tests, the system was able to differentiate between bleeding occurrence and intraoperative irrigation; thus, reducing the risk of false-negative and false-positive results. Conclusion: The current level of AI and AR technologies enables the development of real-time hazard detection systems as efficient assistance tools for surgeons, especially in high-risk interventions.

Unveiling the Evolution of Virtual Reality in Medicine: A Bibliometric Analysis of Research Hotspots and Trends over the Past 12 Years

“…However, the combination of visualization technologies and AI still faces many limitations and obstacles in the face of practical medical problems. For example, the validity of physiological data needs to be improved, the extensibility of methods needs to be evaluated, the current hardware capacity is limited, there is a lack of tactile feedback, data privacy issues are controversial, the application cost of some systems may beyond affordability, and there is still a gap between the progress made by neophytes in simulated operation training and clinical practice; the latter is somewhat risky but the trainers make greater progress [44][45][46][47][48]. This requires the collaboration of doctors, AI engineers, and other professionals to develop a more powerful multi-modal treatment system.…”

Calibration and validation of a bleeding detector for beveled PMMA applicators in electron intraoperative radiotherapy

“…Although bleeding detectors for medical applications can be found in the literature [14][15][16][17], the problem of real-time bleeding detection in IOERT treatments has only been found in [18][19][20]. Bleeding detectors described in [14][15][16][17] basically perform the detection through image processing [14][15][16] or augmented reality and processing through artificial intelligence [17]. However, none of these works specify the accumulated fluid height that is required in this application.…”

“…The use of detectors to measure the presence of fluid in a recipient is common in fields such as in industrial, medical or environmental monitoring applications [10][11][12][13]. Although bleeding detectors for medical applications can be found in the literature [14][15][16][17], the problem of real-time bleeding detection in IOERT treatments has only been found in [18][19][20]. Bleeding detectors described in [14][15][16][17] basically perform the detection through image processing [14][15][16] or augmented reality and processing through artificial intelligence [17].…”