No abstract

Abstract. We present a method for the 3-D shape reconstruction of the retinal fundus from stereo paired images. Detection of retinal elevation plays a critical role in the diagnosis and management of many retinal diseases. However, since the shape of ocular fundus is nearly planar, its 3-D depth range is very narrow. Therefore, we use the location of vascular bifurcations and a plane+parallax approach to provide a robust estimation of the epipolar geometry. Matching is then performed using a mutual information algorithm for accurate estimation of the disparity maps. To validate our results, in the absence of camera calibration, we compared the results with measurements from the current clinical gold standard, optical coherence tomography (OCT).

This study presents methods to 2-D registration of retinal image sequences and 3-D shape inference from fluorescein images. The Y-feature is a robust geometric entity that is largely invariant across modalities as well as across the temporal grey level variations induced by the propagation of the dye in the vessels. We first present a Y-feature extraction method that finds a set of Y-feature candidates using local image gradient information. A gradient-based approach is then used to align an articulated model of the Y-feature to the candidates more accurately while optimizing a cost function. Using mutual information, fitted Y-features are subsequently matched across images, including colors and fluorescein angiographic frames, for registration. To reconstruct the retinal fundus in 3-D, the extracted Y-features are used to estimate the epipolar geometry with a plane-and-parallax approach. The proposed solution provides a robust estimation of the fundamental matrix suitable for plane-like surfaces, such as the retinal fundus. The mutual information criterion is used to accurately estimate the dense disparity map, while the Y-features are used to estimate the bounds of the range space. Our experimental results validate the proposed method on a set of difficult fluorescein image pairs.

In this study, we address the problem of 3-D dense metric reconstruction and registration from multiple images, given that the observed surface is nearly planar. This is difficult, as classical methods work well only if the scene is truly planar (mosaicing) or the scene has certain significant depth variations (classical Structure-from-Motion (SfM)). One domain in which this problem occurs is image analysis of the retinal fundus. Our approach is to first assume planarity, and perform 2-D global registration. A first bundle adjustment is applied to find the camera positions in metric space. We then select two images and compute the epipolar geometry between them using plane+parallax approach. These images are matched to generate a dense disparity map using mutual information. A second bundle adjustment is applied to transform the disparity map into a dense metric depth map, fixing the 2 camera positions. A third bundle adjustment is performed to refine both camera positions and a 3-D structure. All images are back-projected to the 3-D structure for the final registration. The entire process is fully automatic. In addition, a clear definition of "near-planarity" is provided. 3-D reconstruction is shown visually. The method is general, and can be applied to other domains, as shown in the experiments.

We propose a general framework for multiple target tracking across multiple cameras using max-flow networks. The framework integrates target detection, tracking, and classification from each camera and obtains the cross-camera trajectory of each target. The global data association problem is formed as a maximum a posteriori (MAP) problem and represented by a flow network. Similarities of time, location, size, and appearance (classification and color histogram) of the target across cameras are provided as inputs to the network and the target's optimal cross-camera trajectory is found using the max-flow algorithm. The implemented system is designed for real-time process with high-resolution videos (10MB per frame). The framework is validated on high resolution camera networks with both overlapping and non-overlapping fields of view in urban scenes.