Partial 3D-reconstruction of the colon from monoscopic colonoscopy videos using shape-from-motion and deep learning

Abstract

For the image-based documentation of a colonoscopy procedure, a 3D-reconstuction of the hollow colon structure from endoscopic video streams is desirable. To obtain this reconstruction, 3D information about the colon has to be extracted from monocular colonoscopy image sequences. This information can be provided by estimating depth through shape-from-motion approaches, using the image information from two successive image frames and the exact knowledge of their disparity. Nevertheless, during a standard colonoscopy the spatial offset between successive frames is continuously changing. Thus, in this work deep convolutional neural networks (DCNNs) are applied in order to obtain piecewise depth maps and point clouds of the colon. These pieces can then be fused for a partial 3D reconstruction.

Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status

“…Simultaneously, composition of more robust datasets for algorithm training, preferably only containing images/videos that are annotated by multiple experts, could aid in creating a more reliable reference standard. Variability in colon length could possibly be assessed, and accounted for, by using recently developed applications for image depth estimation and topographical reconstruction 52 53 54 , assessment of endoscope camera pose 55 and endoscopic three-dimensional (3D) colon reconstruction 56 57 58 59 . Besides, 3D colon reconstruction 56 57 58 59 techniques might open doors for development of other polyp localization approaches, since these could potentially visualize detected polyps within reconstructions of the complete colon.…”