An FPGA-based real-time occlusion robust stereo vision system using semi-global matching

Cambuim, Lucas F. S.; Oliveira, Luiz Antonio de; Barros, Edna; Ferreira, Antonyus P. A.

doi:10.1007/s11554-019-00902-w

Cited by 35 publications

(24 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Stereo Matching step calculates disparity maps that permit us to estimate the pedestrian's distances. In this step, our FPGA-based module [24] that processes the Semi-Global Matching (SGM) technique [30] in streaming and without frame loss is adopted. This technique propagates throughout the image, matching cost information through four onedimensional paths producing more robust disparity maps for the urban context.…”

Section: Case Study: the Pcp System Heterogeneous Architecturementioning

confidence: 99%

“…In addition to the PD system architecture, another essential contribution to the performance improvement of the PCP system is in the pedestrian distance estimation step that depends on the results of the Semi-Global Matching (SGM) [23]. Since the SGM demands a high computational cost, our high-performance FPGA-based SGM [24] was integrated into the PCP system along with the proposed detector.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FPGA-Based Pedestrian Detection for Collision Prediction System

Cambuim

Barros

2022

Sensors

View full text Add to dashboard Cite

Pedestrian detection (PD) systems capable of locating pedestrians over large distances and locating them faster are needed in Pedestrian Collision Prediction (PCP) systems to increase the decision-making distance. This paper proposes a performance-optimized FPGA implementation of a HOG-SVM-based PD system with support for image pyramids and detection windows of different sizes to locate near and far pedestrians. This work proposes a hardware architecture that can process one pixel per clock cycle by exploring data and temporal parallelism using techniques such as pipeline and spatial division of data between parallel processing units. The proposed architecture for the PD module was validated in FPGA and integrated with the stereo semi-global matching (SGM) module, also prototyped in FPGA. Processing two windows of different dimensions permitted a reduction in miss rate of at least 6% compared to a uniquely sized window detector. The performances achieved by the PD system and the PCP system in HD resolution were 100 and 66.2 frames per second (FPS), respectively. The performance improvement achieved by the PCP system with the addition of our PD module permitted an increase in decision-making distance of 3.3 m compared to a PCP system that processes at 30 FPS.

show abstract

Section: Case Study: the Pcp System Heterogeneous Architecturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FPGA-Based Pedestrian Detection for Collision Prediction System

Cambuim

Barros

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…For real-time processing and embedded capacities, several stereo-matching algorithms in the present study have been implemented using FPGA technology [3,12]. Depending on the configuration of the cameras, the range of disparity levels varies for algorithms implemented inside FPGA devices; this implies an increment of hardware resources consumption.…”

Section: Stereo-matching Algorithms Suitable For Hardware Implementmentioning

confidence: 99%

Real‐time multi‐window stereo matching algorithm with fuzzy logic

Vázquez-Delgado

Pérez-Patricio

Aguilar-González

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

Stereo matching obtains a depth map called a disparity map that indicates or shows the positions of the objects in a scene. To estimate a disparity map, the most popular trend consists of comparing two images (left‐right) from two different points from the same scene. Unfortunately, small window sizes are suitable to preserve the edges, while large window sizes are required in homogeneous areas. To solve this problem, in this article, a novel real‐time stereo matching algorithm embedded in an FPGA is proposed. The approach consists of estimating disparity maps with different window sizes by using the sum of absolute differences (SAD) as a local correlation metric. Once the disparity maps are obtained, the left‐right consistency for each window size is computed. At the end of this stage, the centre pixel deviation is estimated through a 5 × 5 window and the Sobel gradient is extracted from the left image. Finally, both parameters are processed by a Fuzzy Inference System (FIS), which combines the calculated disparities and generates a final disparity map. An architecture embedded in FPGA is established and hardware acceleration strategies are discussed. Experimental results demonstrated that this algorithmic formulation provides promising results compared with the current state of the art.

show abstract

“…The key point of this design is the combination of disparity and multi-level parallelisms such as image line processing to deal with data dependency and data irregular access pattern problems in SGM. Later, to improve the robustness of SGM and achieve a more accurate stereo matching, Cambuim et al [77] combine the sampling-insensitive absolute difference in the pre-processing phase, and propose a novel streaming architecture to detect noisy and occluded regions in the postprocessing phase. The design is evaluated in a full stereo vision system using two heterogeneous platforms, DE2i-150 and DE4, and achieves 25 fps processing rate in 1024×768 HD maps with 256 disparity levels.…”

Section: Semi-global Matching On Fpgamentioning

confidence: 99%

A Survey of FPGA-Based Robotic Computing

Wan

Bo²,

Li³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recent researches on robotics have shown significant improvement, spanning from algorithms, mechanics to hardware architectures. Robotics, including manipulators, legged robots, drones, and autonomous vehicles, are now widely applied in diverse scenarios. However, the high computation and data complexity of robotic algorithms pose great challenges to its applications. On the one hand, CPU platform is flexible to handle multiple robotic tasks. GPU platform has higher computational capacities and easy-touse development frameworks, so they have been widely adopted in several applications. On the other hand, FPGA-based robotic accelerators are becoming increasingly competitive alternatives, especially in latency-critical and power-limited scenarios. With specialized designed hardware logic and algorithm kernels, FPGA-based accelerators can surpass CPU and GPU in performance and energy efficiency. In this paper, we give an overview of previous work on FPGA-based robotic accelerators covering different stages of the robotic system pipeline. An analysis of software and hardware optimization techniques and main technical issues is presented, along with some commercial and space applications, to serve as a guide for future work.

show abstract

An FPGA-based real-time occlusion robust stereo vision system using semi-global matching

Cited by 35 publications

References 29 publications

FPGA-Based Pedestrian Detection for Collision Prediction System

FPGA-Based Pedestrian Detection for Collision Prediction System

Real‐time multi‐window stereo matching algorithm with fuzzy logic

A Survey of FPGA-Based Robotic Computing

Contact Info

Product

Resources

About