Analysis and Implementation of the Semi-Global Matching 3D Vision Algorithm Using Code Transformations and High-Level Synthesis

Abstract

High-level synthesis (HLS) offers several advantages, such as faster simulation run-time and better design re-use, thanks to the higher level of abstraction. This work uses HLS to implement the Semi-Global Matching (SGM) algorithm, which is frequently used in stereo vision systems, e.g. for automotive applications. The hardware implementation is based on a Xilinx ® Virtex 7 FPGA. The initial algorithmic "golden" model used very large arrays, which had to be mapped to an external DRAM and brought into the on-chip RAM of the FPGA on demand. This required both adding the memory transfer loops and inserting calls to the AXI transactors that access the DRAM through the on-chip DDR slave. Moreover, the initial single-threaded algorithm had to be parallelized, by converting the top-level sweeps of the image in eight directions into as many threads. The access to the DRAM was then managed with a centralized controller. This modified SystemC design proved to be suitable to achieve the target real-time performance. The design space was thus explored by making several fairly different micro-architectural choices. In the end, it was possible to obtain an implementation which is comparable to a very efficient (and hence very inflexible) manual RTL design that had been previously developed, including a very sophisticated fine-grained management of data and computation.

Model Based Design of a Real Time FPGA-Based Lens Undistortion and Image Rectification Algorithm for Stereo Imaging

“…Other researchers have proposed various solutions to speed up the development process to still benefit from the speed and flexibility of FPGAs. Several researchers have proposed using C-like languages such as Michalik et al and Qamar et al, that can be compiled to HDL [2] [16]. Although C-like languages have been proven to speed up the initial development cycle, the approach only improves the coding process and not the simulation and verification process.…”

Autonomous stereo vision system for depth computation of moving object

“…This work achieved very good results, but for resolution 320x240 with only 32 disparity levels. Others like [4,5,6] applied Semi-Global Matching (SGM) methods by Census Transform and Hamming Distance as main matching approaches.…”

“…These issues can be solved through a rectification process, which uses the parameters of each camera to undistort each image, and then, the parameters from the stereo camera set to rectify the images planes [5,6,8,12]. These parameters are predetermined by a calibration process, in which a regression method is used to determine the distortion of a predefined pattern, usually a checkerboard.…”

“…Semi-Global Matching [5,6,22] effectively combines ideas of global and local algorithms for accurate pixel-wise matching at low runtime. The first SGM implementations used Mutual Information (MI), which is very suitable for unrectified images, because the transformations are irrelevant when only individual pixels are considered.…”

“…5 presents the flow of configuring the horizontal resolution, the eol (end of line) computation, and the sof (start of frame) detection. The register address is initialized with zero when vsync_registered is not active for blank interval between frames.…”