Real-time depth processing for embedded platforms

Abstract

For many applications in low-power, real-time robotics, stereo cameras are the sensors of choice for depth perception. Their biggest drawback, however, is that they do not directly sense depth maps; instead, these must be estimated through data-intensive processes. Motivated by applications in space and mobile robotics, we implement and evaluate an FPGA-accelerated adaptation of the ELAS algorithm. Despite offering one of the best tradeoffs between efficiency and accuracy, ELAS has only been shown to run at 1.5 − 3 fps on a high-end CPU. Our system preserves all intriguing properties of the original algorithm, such as the slanted plane priors, but can achieve a frame rate of 47fps whilst consuming under 4W of power. Unlike previous FPGA based designs, we take advantage of both components on the CPU/FPGA System-on-Chip to showcase the strategy necessary to accelerate more complex and computationally diverse algorithms for such low power, real-time systems.

Real-Time Dense Stereo Matching With ELAS on FPGA-Accelerated Embedded Devices

“…5, unlike the CT that reuses previously extracted features, the SADs must be recomputed every time. Also, SAD implementations that achieve similar throughput, such as the one in [36], require an additional window buffer to store previous column SAD computations (bottom of Fig. 5).…”

R3SGM: Real-Time Raster-Respecting Semi-Global Matching for Power-Constrained Systems

“…It has thus proved highly popular in real-world systems, and many FPGA-based approaches have been inspired by it [2], [10], [20], [28], [41], [42]. Other FPGA-based methods have also been presented [34], [36], [37], [46], [50], but, whilst typically faster than those inspired by SGM, they seldom reach the same level of accuracy. However, because the disparities that SGM computes for neighbouring pixels are based on star-shaped sets of input pixels that are mostly disjoint, SGM suffers from streaking in areas in which the data terms in some directions are weak, whilst those in other directions are strong.…”