Improving the Accuracy of Feature Extraction for Flexible Endoscope Calibration by Spatial Super Resolution

Abstract

Many applications in the domain of medical as well as industrial image processing make considerable use of flexible endoscopes - so called fiberscopes - to gain visual access to holes, hollows, antrums and cavities that are difficult to enter and examine. For a complete exploration and understanding of an antrum, 3d depth information might be desirable or yet necessary. This often requires the mapping of 3d world coordinates to 2d image coordinates which is estimated by camera calibration. In order to retrieve useful results, the precise extraction of the imaged calibration pattern's markers plays a decisive role in the camera calibration process. Unfortunately, when utilizing fiberscopes, the image conductor introduces a disturbing comb structure to the images that anticipates a (precise) marker extraction. Since the calibration quality crucially depends on subpixel-precise calibration marker positions, we apply static comb structure removal algorithms along with a dynamic spatial resolution enhancement method in order to improve the feature extraction accuracy. In our experiments, we demonstrate that our approach results in a more accurate calibration of flexible endoscopes and thus allows for a more precise reconstruction of 3d information from fiberoptic images.

Endomicroscopy for Computer and Robot Assisted Intervention

“…Among existing computational super-resolution methods, the most promising developments include: learning prior information by example or via parsimony in some basis [65]; mixed L1/L2 optimisation methods to promote this sparsity [66]; hierarchical variational Bayesian methods to approximate the posterior, providing hyper-parameter estimation and guiding optimisation under uncertainty [67]; integration of registration uncertainty [68] or blind deconvolution [69]. In fibre endomicroscopy, work has been limited to improving camera calibration by employing Delauney triangulation based interpolation (with no deblurring step) on test patterns [70]. They present the two comb structure algorithms and the spatial super resolution model to simulate fiberscopic transmission for evaluation.…”

Texture-based computer-assisted diagnosis for fiberscopic images

“…To evaluate the effects of the fiberscopic filtering algorithm To simulate the effect of fiberscopic image acquisition a method previously presented by Rupp et al [15], [16] has been used. This method uses a reference image from a fiberscopic system to capture the fiber structure by locating the fiber centers first.…”

Image computing for fibre-bundle endomicroscopy: A review

“…Throughout the years, a number of approaches employing band-pass filtering in the Fourier domain have been proposed (Dickens et al, 1998;Dickens et al, 1997Dickens et al, , 1999Dumripatanachod and Piyawattanametha, 2015;Ford et al, 2012b;Han et al, 2010;Lee and Han, 2013b;Maneas et al, 2015;Rupp et al, 2007;Winter et al, 2006).…”

“…Hence, by effectively aligning the imaged structures and combining a sequence of shifted frames (i) the honeycomb structure can be suppressed/eliminated, (ii) the imaging resolution can be increased. Numerous attempts have examined the effect of different shift patterns, altering the location of the core pattern with respect to the imaged structure, and superposition methods, such as deriving the average or maximum intensity of the aligned images on fibreoscopic images (Kyrish et al, 2010;Lee and Han, 2013a;Lee et al, 2013;Rupp et al, 2007). Alternatively, (Cheon et al, 2014a, b;Vercauteren et al, 2006;Vercauteren et al, 2005) employed the random movements during data acquisition, as would be expected in a realistic clinical scenario, to create an enhanced composite image.…”