Hardware-software implementation of the PointPillars network for 3D object detection in point clouds

Stanisz, Joanna; Lis, Konrad; Kryjak, Tomasz; Gorgon, Marek Boguslaw

doi:10.1145/3441110.3441150

Cited by 2 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The FPGA part takes 262 milliseconds to finish, but other operations on PS part are expensive too. In [17] the FPGA part execution time was 1.99 seconds. It was reduced by:…”

Section: Hardware-software Implementation Of the Car Detection Systemmentioning

confidence: 99%

See 1 more Smart Citation

Implementation of the PointPillars Network for 3D Object Detection in Reprogrammable Heterogeneous Devices Using FINN

Stanisz

Lis

Gorgon

2021

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The Brevitas / PyTorch tools were used for network quantisation (described in our previous paper) and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on a heterogeneous embedded platform with maximum 19% AP loss in 3D, maximum 8% AP loss in BEV and execution time 375ms (the FPGA part takes 262ms). We have also compared our solution in terms of inference speed with a Vitis AI implementation proposed by Xilinx (19 Hz frame rate). Especially, we have thoroughly investigated the fundamental causes of differences in the frame rate of both solutions. The code is available at https://github.com/vision-agh/pp-finn.

show abstract

“…The FPGA part takes 262 milliseconds to finish, but other operations on PS part are expensive too. In [17] the FPGA part execution time was 1.99 seconds. It was reduced by:…”

Section: Hardware-software Implementation Of the Car Detection Systemmentioning

confidence: 99%

“…This article is an extension of the conference paper [17] presented at DASIP'21 workshop in January 2021. We conducted more experiments with the PointPillars implementation in FINN -using the newest version 0.5.…”

Section: Introductionmentioning

confidence: 99%