FADEC: FPGA-based Acceleration of Video Depth Estimation by HW/SW Co-design

Hashimoto, Nobuho; Takamaeda-Yamazaki, Shinya

doi:10.1109/icfpt56656.2022.9974565

Cited by 3 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the proposed algorithm is fast and achieves lower power consumption, it adds some constraints on the direction of motion and speed, which makes it unsuitable for complex environmental conditions. [6] presented an HW/SW co-design approach to implement DeepVideoMVS [13] network on FPGA which achieves 60 times faster inference speed compared to CPU with the cost of slight accuracy degradation. [5] proposes FasterMDE, an optimized encoder-decoder network that uses multiobjective neural architecture search to optimize the encoder block for edge implementation.…”

Section: Related Workmentioning

confidence: 99%

FPGA Implementation of CNN Based DepthEstimation Network: MiDaSNet

Adiyaman,

Baskaya

2024

Preprint

View full text Add to dashboard Cite

Real-time depth estimation is crucial in many vision-related tasks, including autonomous driving, 3D reconstruction, and SLAM applications. In recent years, many methods have been proposed to solve depth maps from images by utilizing different modality setups like monocular vision, binocular vision, or sensor fusion. However, for real-time deployment on edge devices, complex methods are not suitable due to latency constraints and the limited computation capacity. For edge implementation, the models should be simple, minimal in size, and also hardware-friendly. Considering these factors, we implemented MiDaSNet, which works on the simplest setup of monocular vision and utilizes hardware-friendly CNN-based architecture, for real-time depth estimation on the edge. Besides, since the model is trained on diverse datasets, it shows stable performance across different mediums. For edge implementation, we quantized the model weights down to an 8-bit fixed-point representation. Then, we deployed the quantized model on an inexpensive FPGA card, Kria KV260, utilizing predefined deep-learning processing units embedded in the programmable logic. The results show that our quantized model achieves 82.6% zero-shot accuracy on the NYUv2 dataset with 50.7 fps inference speed on the card.

show abstract

Section: Related Workmentioning

confidence: 99%