Video-endoscopic image analysis for 3D reconstruction of the surgical scene

Abstract

-Video-endoscopic image analysis is an important source of information for extracting relevant 3D information of the surgical scene in minimally invasive surgery. This work proposes a new approach for getting depth information and relative position between instruments and organs based on video image analysis. The proposed image analysis extracts information from: (1) the projective geometry of the surgical setting in the image to determine laparoscopic tool's pose, and (2) the scene illumination captured in the image for getting a depth-map of the surgical scene. Results have verified the potential of video-analysis based methods for obtaining 3D information of the whole surgical scene. This information offers new possibilities for developing new augmented reality surgical applications and for improving surgical training with objective metrics. The main benefit of this approach is the lack of extra elements which disturb surgeons' performance in the clinical routine.

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

“…The majority of reconstruction approaches in laparoscopy use EKF-based SLAM methods for scene reconstruction (Grasa et al [58]- [60] and Mountney et al [108]), and these have also been extended to handle tissue deformation [106], [109]). Gonzalez et al [56] have combined SfS to estimate a depth map with a tracking approach for pose estimation of surgical tools. The few publications on stitching and reconstruction of the colon are dominated by the classical SfM approach (Thormaehlen et al [144] (2002), Koppel et al [82] (2007), Chen et al [39]).…”