3-D Metric Reconstruction and Registration of Images of Near-planar Surfaces

Abstract

In this study, we address the problem of 3-D dense metric reconstruction and registration from multiple images, given that the observed surface is nearly planar. This is difficult, as classical methods work well only if the scene is truly planar (mosaicing) or the scene has certain significant depth variations (classical Structure-from-Motion (SfM)). One domain in which this problem occurs is image analysis of the retinal fundus. Our approach is to first assume planarity, and perform 2-D global registration. A first bundle adjustment is applied to find the camera positions in metric space. We then select two images and compute the epipolar geometry between them using plane+parallax approach. These images are matched to generate a dense disparity map using mutual information. A second bundle adjustment is applied to transform the disparity map into a dense metric depth map, fixing the 2 camera positions. A third bundle adjustment is performed to refine both camera positions and a 3-D structure. All images are back-projected to the 3-D structure for the final registration. The entire process is fully automatic. In addition, a clear definition of "near-planarity" is provided. 3-D reconstruction is shown visually. The method is general, and can be applied to other domains, as shown in the experiments.

Retinal image registration from 2D to 3D

“…The 3D registration process also needs accurate 3D surface to infer point transformation between images. The reconstruction of retinal images belongs to the category of near-planar surface reconstruction, which is carefully studied in [6]. It is a difficult problem due to the lack of depth information, which is a quasidegenerate case for the estimation of the 3D structure [8].…”

“…The novelty of our approach is the 4-pass bundle adjustment in which the objective is to estimate the poses of all cameras. In [6], the camera selection strategy does not take the baseline into account, and produces poor results when two cameras are close.…”

“…Finally, we use all-pair shortest path algorithm on the set of images [6]. The image with the least total shortest path cost to all the other images is selected as the reference frame.…”

“…and image I i is registered to I ref by Although our proposed approach looks similar to [6], they are different in the following aspects and ours produces significantly better results:…”

Automatic 3D change detection for glaucoma diagnosis

“…First, traditional stereo matching algorithms have great difficulties in matching fundus images because of low-texture, lowcontrast, image blur, non-Lambertian reflectance, and noise from the illumination conditions [10]. Second, most of the retinal surface in a fundus image is nearly planar except the optic disc area [1,4]. The scene of a flat plane is a degenerate case for estimating epipolar geometry.…”