Active stereo: integrating disparity, vergence, focus, aperture and calibration for surface estimation

Abstract

Abstract-Much research has emphasized stereo disparity as a source of depth information. To a lesser extent, camera vergence and lens focus have also been investigated for their utility in depth recovery. Each of these visual cues exhibits shortcomings when used individually in the sense that none alone can be used to reconstruct surfaces for real scenes that often cover a wide field of view and a large range of depth. This paper presents an approach to integration of these cues that attempts to exploit their complementary strengths and weaknesses through active control of camera focus and orientations. In addition, the aperture and zoom settings of the cameras are controlled. The result is an active vision system that dynamically and cooperatively interleaves image acquisition with surface estimation. A dense composite map of a single contiguous surface is synthesized by automatically scanning the surface and combining estimates of adjacent, local surface patches. This problem is formulated as one of minimizing a pair of objective functions. The first such function is concerned with the selection of a target for fixation. The second objective function guides the surface estimation process in the vicinity of the fixation point. Calibration parameters of the cameras are treated as variables during optimization, thus making camera calibration an integral, flexible component of surface estimation. An implementation of this method is described, and a performance evaluation of the system is presented. An average absolute error of less than 0.15% in estimated depth was achieved for a large surface having a depth of approximately 2 m.Index Terms-Active vision, camera calibration, fixation, range from focus, range from stereo, range from vergence, surface estimation, visual cue integration, visual target selection.

<title>Vergence control system for stereo depth recovery</title>

“…In different bibliograpy 3,6,7 this type of interrelationships are shown. Therefore, the vergence system permits the sensor system to be concentrated in interest areas to obtain a greater precision and confidence in the depth measure.…”

“…The goal is to calculate this error from visual information and to execute a control action that make it null, that is to say, to achieve that the intersection of the two focal axes stay over the surface of the object. Different techniques can be found in the literature to generate an estimation of this vergence error 6,12,13,14 .…”

Multiresolution 3-D range segmentation using focus cues

“…For a thin lens, if a point on an object is focused, then the Gaussian lens law holds [6], [9], [12], [15]: (1) where is the distance of the point from the lens, is the image plane distance from the lens, and is the focal length of the lens.…”

“…is an attractive source for depth information, since it has no correspondence problem as in stereo [15]. Focus cues can be efficiently used for measuring depth when the initial ranges are not available, for example, in a multimodal active vision system.…”

Reciprocal-Wedge Transform in Active Stereo

“…Krotkov's system demonstrates the reliability in ranging upon fusion of the focusing and stereo vergence components. Abbott and Ahuja [10] took integration of visual cues to great length in their University of Illinois Active Vision System.…”