BICOS—An Algorithm for Fast Real-Time Correspondence Search for Statistical Pattern Projection-Based Active Stereo Sensors

Abstract

Pattern projection-based 3D measurement systems are widely used for contactless, non-destructive optical 3D shape measurements. In addition, many robot-operated automation tasks require real-time reconstruction of accurate 3D data. In previous works, we have demonstrated 3D scanning based on statistical pattern projection-aided stereo matching between two cameras. One major advantage of this technology is that the actually projected patterns do not have to be known a priori in the reconstruction software. This allows much simpler projector designs and enables high-speed projection. However, to find corresponding pixels between cameras, it is necessary to search the best match amongst all pixels within the geometrically possible image area (that is, within a range on the corresponding epipolar line). The well-established method for this search is to compare each candidate pixel by temporal normalized cross correlation of the brightness value sequences of both pixels. This is computationally expensive and interdicts fast real-time applications on inexpensive computer hardware. We show two variants of our algorithm “Binary Correspondence Search” (BICOS), which solve this task in significantly reduced calculation time. In practice, our algorithm is much faster than traditional, purely cross-correlation-based search while maintaining a similar level of accuracy.

Real-Time FPGA Accelerated Stereo Matching for Temporal Statistical Pattern Projector Systems

“…The scope of this paper is limited to aperiodic sinusoidal temporal pattern projection. Currently, some publications are evaluating the use of such methods [ 4 , 5 , 6 , 8 ]. However, the main subject of these publications mostly stems from the field of optics.…”

“…However, the main subject of these publications mostly stems from the field of optics. The closest research work that addresses the issue of computational complexity in temporal statistical pattern stereo was published by Dietrich et al [ 6 ]. The main contribution of this work was the incorporation of an extended census transform over the temporal axis and the use of the Hamming distance as the matching cost metric when calculating disparity.…”

“… Temporal Census Transform—The temporal variation of brightness at a pixel level is encoded using a modified census transform operation, forming the feature vector, that can be represented as a bitstring. Grayscale pixel values (brightness) are encoded as a bitstring in the same way as that published in [ 6 ]. Disparity Calculator—Using the temporally encoded features (bitstrings), both left-to-right and right-to-left disparities are calculated.…”

“…The drawback of aperiodic sinusoidal fringe projection systems is the increased depth reconstruction error for the same number of temporal measurements, compared to periodic fringe projection methods [ 5 ]. Another less commonly addressed issue is the computational burden associated with the algorithms used for disparity calculation, as highlighted in [ 6 ]. Compared to algorithms used in periodic fringe projection [ 1 ], the computational requirements of temporal pixel-to-pixel correlation limit the portability of such approaches to high-end computing devices.…”

Enhanced Contactless Vital Sign Estimation from Real-Time Multimodal 3D Image Data

“…This 3D sensor works at 850 nm and is thus irritation-free. Using a set of 10 projected light patterns, a 3D point cloud can be calculated using the stereo image correlation method in [ 23 ] with the acceleration algorithm in [ 24 ]. With the use of a graphics processing unit (GPU), a 3D frame rate up to 30 Hz can be realized, giving a real-time 3D video.…”