FPGA Design of Real-Time MDFD System Using High Level Synthesis

Abstract

3D shape information is one of the very important clues in image processing and computer vision. Unlike traditional multi-input depth from defocus (DFD) technique, monocular DFD (MDFD) algorithm proposed by Hu and Haan can reconstruct 3D shape only from a single monocular defocus image with low computing complexity. In this paper, we present a real-time MDFD system implemented on the FPGA device. In order to reduce the FPGA design cost, vivado high level synthesis (VHLS) is applied to design the MDFD system. The system architecture on the basis of FIFO based convolution is first designed through C/C++ code that is further converted to the FPGA design by VHLS. Then the PIPELINE, LOOP_MERGE, and ARRAY_PARTITION directives are used to optimize the latency and interval of the proposed system. The performance and resource utilization of the whole system are evaluated by processing defocus images from the real scene with 640×480 pixel size. The system can process about 22 images at 20 MHz working frequency and keep the 93.29% depth accuracy on the 3D objects test, which achieves a real-time state-of-the-art MDFD system by comparing to other recent works. INDEX TERMS 3D reconstruction, FPGA, monocular depth from defocus, vivado high level synthesis.

Automatic Focusing Method of Microscopes Based on Image Processing

“…is article looks at the principle of autofocus to see if it can be used on a microscope. [11][12][13] is a method to obtain the depth information of the focus target through the defocused image so as to complete the autofocus. DFD method needs to obtain 2-3 frames of images with different defocusing degrees.…”

Real-Time Embedded Eye Image Defocus Estimation for Iris Biometrics

“…Since, in our scenario, it is possible to assume the isotropic behaviour of the PSF and that the blur is due to bad focusing, this approach based on convolution kernels applied in the image domain is fully valid [21,25]. Furthermore, the convolution filtering of digital images can be efficiently addressed using FPGA devices [7,27,28].…”

“…This is not the situation when working with an FPGA. If the input image and convolution kernel are small in value (e.g., Wei et al [7] work with 640 × 480 pixel images and a convolution kernel of 3 × 3), FIFO (first in-first out) memories can be implemented to store the necessary image rows (3 FIFO memories with a depth of 637 each in the case of Wei et al's implementation). In our case, the EMERALD 16MP sensor provides images of 4096 × 4096 pixels, and the design will need 8 FIFO memories with a depth of 4088 (4096 − 8) each.…”

“…Moreover, if the traditional development approach of FPGAs using low-level hardware languages (such as Verilog and VHDL) is usually time-consuming and very inefficient, the use of high-level language synthesis (HLS) tools allows developers to program hardware solutions using C/C++ and OpenCL. This significantly improves the efficiency in FPGA developments [7,8]. Finally, FPGAs are nowadays integrated in multi-processor system-on-chips (MPSoC), in which computer and embedded logic elements are combined.…”

“…The whole framework was mainly built using Vitis HLS and synthesised in the aforementioned AMD/Xilinx UltraScale+ MPSoC (multiprocessor system-on-chip). Given the characteristics of an FPGA, the design option selected for this defocus blur evaluation module was based on convolution kernels [7,9].…”