Resource‐efficient FPGA implementation of perspective transformation for bird's eye view generation using high‐level synthesis framework

Abstract

Bird's eye view (BEV) generation from front-looking video stream is considered an important pre-processing task in various computer vision applications such as driver assistance systems. In this work, hardware implementation of this process using high-level synthesis in Simulink environment has been considered for rapid prototyping under real-time constraints. Traditionally, researchers have employed lookup table-based approaches to circumvent the exorbitant cost of implementing arithmetic modules associated with the perspective transformation. The hardware implementation scheme proposed here, however, demonstrates that a polynomial approximation over the limited domain of the involved operands not only saves precious hardware resources but also provides better fixed-point precision. Synthesis results on Zynq-7000 FPGA show that the proposed circuit reduces the block memory utilisation by 9% compared to the lookup table-based built-in Simulink Vision HDL block. The proposed design evaluates the results in fixed-point format which is essential for subsequent bilinear interpolation to produce high-fidelity output frame, albeit at the cost of 4% increase in DSP48E utilisation. The approximation error of the proposed solution is less than quarter-pixel on average. The proposed hardware has been integrated as an IP core in a hardware-software co-design system. The whole framework is publicly available to facilitate practitioners and researchers.

Model Based Design of a Real Time FPGA-Based Lens Undistortion and Image Rectification Algorithm for Stereo Imaging

“…An integrated tool flow for simulating and generating HDL for computer vision algorithms was created in 2019 with the advent of the Vision HDL Toolbox which allows one to convert images composed of rows and columns into pixel streams that can be processed at the pixel line rates. Bilal et al demonstrated the use of this new toolbox by implementing a perspective transformation for bird's eye view generation [18].…”

Resource-Efficient Hardware Implementation of Perspective Transformation Based on Central Projection

“…Moreover, the hardware design scheme is given, which can solve the optical distortion and perspective distortion to a certain extent. The homography transformation method is used in [19][20][21], which needs to be realized by matrix multiplication, and different transformation matrices can realize different perspective transformations. The result obtained by this method is more in line with the observation habits of the human eye, but the solution of the transformation matrix needs to solve the 8-variable linear equation system, which is extremely difficult for hardware.…”

“…Moreover, Refs. [19][20][21] are all implemented on the Field Programmable Gate Array (FPGA), which are used to generate the Bird's-Eye View (BEV), correction of input video from two cameras, and obstacle detection, respectively.…”

“…In the papers mentioned above, Refs. [9,18] use linear average stretching, and [1,[5][6][7][8][10][11][12][13][14][15][16][17][19][20][21] use homography transformation. However, the homography transformation requires the use of a transformation matrix, but the transformation matrix contains eight unknowns, and its solution is very complicated.…”

“…Most of the existing schemes use a pure software approach to achieve homography transformation [1,[5][6][7][8][10][11][12][13][14][15][16][17]. In addition, a small number of papers present a solution to implement the homography transformation in hardware [19][20][21], but they all require software assistance to calculate the transformation matrix. This paper proposes a hardware design of perspective transformation based on central projection.…”