What is the Future of Minimally Invasive Sinus Surgery: Computer-Assisted Navigation, 3D-Surgical Planner, Augmented Reality in the Operating Room with ‘in the Air’ Surgeon’s Commands as “Biomechanics” of the New Era in Personalized Contactless Hand-Gesture Non-Invasive Surgeon-Computer Interaction?

Abstract

Purpose: We were focused on the development of personal-3D-navigation system and application of augmented reality in the operating room per viam personalized contactless hand-gesture non-invasive surgeon-computer interaction, with higher intraoperative safety, reduction of operating time, as well as the length of patient postoperative recovery.Methods: Simultaneous use of video image, 3D anatomic fields and navigation in space, with the application of our original special plug-in application for OsiriX platform, enabling users to use LM-sensor as an interface for camera positioning in 3DVR and VE views, and integrating speech recognition as a VC solution, in an original way.Results: Management of image 2D-3D-video-medical documentation, as well as the control of marker-based virtual reality simulation in real time during real operation with per viam our personalized contactless "in the air" surgeon's commands. Conclusion:The use of modern technologies in head and neck surgery in the last 30 years (e.g., FESS, NESS, and robotic surgery) has enabled surgeons to demonstrate spatial anatomic elements in the operating field, which was quite inconceivable before. This

Do We Really Need a New Navigation-Noninvasive “on the Fly” Gesture-Controlled Incisionless Surgery?

“…The best way to predict the future is to create it ourselves. Today, our understanding of the anatomy in humans could be deceived, where we can replace the true reality with the simulated reality [1][2][3] that enables precise, safer and faster diagnosis [4], as well as surgery, creating an impression of another 'external' world around the man [5,6]. Through the concept of simulated reality, almost the same environment can be achieved as in true reality.…”

“…Traditionally, simulated reality is considered as an extension of virtual reality (VR), which is widely used in different telemedicine (TM) platforms, but in our work (Figure 1), we would like to elaborate the need of improving these VR tools in order to get better user experience both in preoperative virtual analysis and during the surgery. With all well-known benefits of adopting VR tools in surgeries, such as better planning, high quality data analysis and simulation, we would like to introduce our original proposal of touchless controlling VR tools in operation room (OR) to come as close as possible to the concept of simulated reality, as we have reported previously [2,5]. open source, operation system agnostic, approved for medical use and independent of hardware.…”

“…Comparison with previous doctrine in human medicine clearly indicates that both preoperative/intraoperative manipulation with three-dimensional-volume rendering slices of the human anatomy per viam touchless surgical navigation system with simulation of virtual activities has become reality in the operation room. Navigating through narrow pathways in VE, we noticed that the camera could stay in the tissue [2,5], thus enabling a substantially different understanding of spatial relations (as exemplified by the nose/sinuses) between 2D and 3D images of human anatomy.…”

“…That is why in our activities we employed the following: a. Pre-and postoperatively the most widely used Digital Imaging and Communications in Medicine (DICOM), with advanced post-processing techniques in two-dimensional (2D)/three-dimensional (3D), as well as four-dimensional (4D) navigation (OsiriX MD); b. Hardware (HD)-sensor device that supports hand and finger motions as input, thus requiring no hand contact or touching (RealSenseIntelDepthCamera and/or Leap Motion (LM Inc., San Francisco, CA, USA); and c. Our original, specially designed software (SW) that integrates LM-controller with medical imaging systems [2,5] completely different from some products already described in medical literature (Figure 2). Rather than the fictional virtual world (VW), our 3D volume rendering (VRen) solution is based on real inputs based on gesture control and manner of movements, enhancing the simple VRen with the real needs according to the awareness of medical specialist.…”

“…We have minimized surgeon distraction, intervention time and anesthesia time while interacting with the positioning system per viam 'different types of gestures', and our special original plug-in (designed by our Bitmedix IT team; www. bitmedix.com) application for DICOM viewer; "navigation through" in the VW, with the use of LM sensor as an interface for camera positioning in 3D-virtual endoscopy (VE)/virtual surgery (VS) [1,2] integrating speech recognition as a voice command (VC) solution in an original way [5]. Our gestures provided very easily the interaction with the real-world elements, pre-and intraoperatively, per viam natural movements while performing navigation (VE/ VS) in the patient's head during the surgery/telesurgery (TS) [2] (Figure 4).…”