Telepresence for surgical assistance and training using eXtended reality during and after pandemic periods

Abstract

Existing challenges in surgical education (See one, do one, teach one) as well as the COVID-19 pandemic make it necessary to develop new ways for surgical training. Therefore, this work describes the implementation of a scalable remote solution called “TeleSTAR” using immersive, interactive and augmented reality elements which enhances surgical training in the operating room. The system uses a full digital surgical microscope in the context of Ear–Nose–Throat surgery. The microscope is equipped with a modular software augmented reality interface consisting an interactive annotation mode to mark anatomical landmarks using a touch device, an experimental intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics. The new educational tool was evaluated and tested during the broadcast of three live XR-based three-dimensional cochlear implant surgeries. The system was able to scale to five different remote locations in parallel with low latency and offering a separate two-dimensional YouTube stream with a higher latency. In total more than 150 persons were trained including healthcare professionals, biomedical engineers and medical students.

Artificial Intelligence-Based Algorithms in Medical Image Scan Segmentation and Intelligent Visual Content Generation—A Concise Overview

“…The segmented structures (in this case, organs and their disorders) may be next applied to the development of the 3D virtual environment [141]. These 3D objects may be implemented through, for example, holograms displayed in a head-mounted display (HDM), such as mixed-reality glasses in medical diagnostics [150], preoperative imaging [151], surgical assistance [152,153], robotics surgery [154], and medical education [87,88]. However, the crucial issue is connected with the quality of the obtained segmented structures, and this process can be significantly accelerated and improved by the use of artificial intelligence.…”

Exploring Augmented Reality Integration in Diagnostic Imaging: Myth or Reality?

“…We can highlight several key points regarding the impact of the COVID-19 pandemic on the development of AR applications in healthcare, most of the themes of which relate to diagnostic imaging, as follows [23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38] Exploring new opportunities in healthcare that have never been faced or thought of before [23].…”

“…The compilation of studies [23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38] presents a rich tapestry of potential applications emerging from current research endeavors.…”

Artificial Intelligence-Based Algorithms in Medical Image Scan Segmentation and Intelligent Visual-Content Generation

“…The segmented structures (in this case organs and their disorders) may be next applied to the development of the 3D virtual environment [105]. These 3D objects may be implemented through for example, holograms displayed in the head-mounted display (HDMs) like Mixed Reality glasses in medical diagnostics [113], pre-operative imaging [114], surgical assistance [115,116], robotics surgery [117], and medical education [81,82]. However, the crucial issue is connected with the quality of obtained segmented structures, and this process can be significantly accelerated and improved by the use of Artificial Intelligence.…”