Endoguide: Endoskop mit 360°-Ansicht

Abstract

Over the last decades, minimally invasive surgery (MIS) has become more and more important, although surgeons still have to deal with limited orientation and difficult navigation. The 2 D camera image of the endoscope provides the only visual feedback. However, the field of view is narrow and the surgeon cannot control the viewing direction himself because the endoscope is usually held by an assistant. The "Endoguide" project, founded by the German Federal Ministry of Education and Research, aims to develop a novel computer-assisted surgery (CAS) system for laparoscopic interventions, consisting of two major parts: a new type of endoscope with variable viewing direction and a processing unit for offering VR / AR support and intuitive user input paradigms. Instead of being held by an assistant, the new endoscope will be mounted and equipped with electric motors for controlling the viewing direction, focus and zoom of the camera. The CAS unit enables a multitude of advanced features. The entire system including the endoscope can be controlled through several touchless interfaces, such as speech recognition, head or gaze tracking. Moreover, real-time GPU-based video processing and optical tracking facilitate several automatic features that range from a FFT-based auto focus to image-based instrument tracking and adaptive region-of-interest selection. In particular the latter allows the surgeon to concentrate on his tasks by automatically providing the correct view. In order to overcome the endoscope's limited field of view, the system automatically captures and stitches 360 degrees panoramic overview scans, which can be viewed on a regular screen or in immersive environments like small dome projections. The current live-view from the endoscope is displayed as an inset at the correct position within the generated panoramic still image. Data and annotations from pre-surgical planning (e.g. from CT / MRI) can be overlaid and provide valuable information during the intervention. Tracking the head of the surgeon allows to directly couple the viewing direction inside the dome with the orientation of the endoscope and, hence, to provide a direct visual feedback