Real-time, 3-D multi object position estimation and tracking

Abstract

For autonomously acting robots and driver assistance systems powerful optical stereo sensor systems are required. Object positions and environmental conditions have to be acquired in real-time. In this paper a hardware-software co-design is applied, acting within the presented stereophotogrammetric system. For calculation of the depth map an optimized algorithm is implemented as a hierarchical parallel hardware solution. By adapting the image resolution to the distance, real-time processing is possible. The object clustering and the tracking is realized in a processor. The density distribution of the disparity in the depth map (disparity histogram) is used for object detection. A Kalman filter stabilizes the parameters of the results.

Stereoscopic imaging for obstacle detection onboard low-flying unmanned aerial vehicles

“…As well, there are inherent optical differences that arise when sensors are separated by a significant baseline distance that may pose difficulties in the correlation process [21]. The correlation procedure is also very computationally intensive and is considered to be the main 'bottleneck' of any optical sensor obstacle detection system in both stereoscopic analysis as well as optical flow analysis [21], [24], [31][32][33].…”

“…f uv , is the mean of the image in the region under the window segment and t is the mean of the segment itself. Other area correlation techniques can also be used for this purpose [31].…”

“…The accuracy of the disparity measurement depends, therefore, on how well the correlation function is defined. If the correlation function yields no obvious maximum peak then the disparity value is not reliable [31].…”

“…Where / is the focal length of the sensors, B is the baseline separation between them, q is the pixel size and d is the disparity [34], [31], [36]. In practice this equation is seldom applied in this form due to varying camera and lens models and instead a calibration process is used to determine the parameters.…”

“…Some systems have attempted to control the stereoscopic complexity by implementing the correlation in hardware such as a field-programmable gate array (FPGA). Using this method makes the computational time required by the correlation 'short' and 'deterministic' [31].…”