PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection

Shi, Guangsheng; Li, Ruifeng; Ma, Chao

doi:10.1007/978-3-031-20080-9_3

Cited by 93 publications

(60 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Without loss of generality, we follow the framework of CenterPoint-Pillar [48] and append our DSVT before BEV backbone. Besides that, we also follow [16,21,31] that uses IoU-rectification scheme to incorporate the IoU information into confidence scores. Two-stage model.…”

Section: Detector Setupmentioning

confidence: 99%

“…We adopt the same learning rate scheme as [48]. During inference, following [16,31], we use class-specific NMS with the IoU threshold of 0.7, 0.6 and 0.55 for vehicle, pedestrian and cyclist, respectively. All inference times are profiled on the same workstation (single NVIDIA A100 GPU and AMD EPYC 7513 CPU) and environment (Ubuntu-18.04, PyTorch-1.10.2, CUDA-11.3).…”

Section: Implementation Detailsmentioning

confidence: 99%

“…Our DSVT follows the framework of CenterPoint [48], which applies a heatmap loss and regression loss to supervise object classification and bounding box estimation. Besides that, we also adopt the same IoU score loss as [16,21,31] to incorporate the IoU information into the confidence scores. The final loss then becomes:…”

Section: B4 Loss Functionmentioning

confidence: 99%

See 2 more Smart Citations

DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

Wang¹,

Chen²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Designing an efficient yet deployment-friendly 3D backbone to handle sparse point clouds is a fundamental problem in 3D object detection. Compared with the customized sparse convolution, the attention mechanism in Transformers is more appropriate for flexibly modeling long-range relationships and is easier to be deployed in real-world applications. However, due to the sparse characteristics of point clouds, it is non-trivial to apply a standard transformer on sparse points. In this paper, we present Dynamic Sparse Voxel Transformer (DSVT), a single-stride windowbased voxel Transformer backbone for outdoor 3D object detection. In order to efficiently process sparse points in parallel, we propose Dynamic Sparse Window Attention, which partitions a series of local regions in each window according to its sparsity and then computes the features of all regions in a fully parallel manner. To allow the crossset connection, we design a rotated set partitioning strategy that alternates between two partitioning configurations in consecutive self-attention layers. To support effective downsampling and better encode geometric information, we also propose an attention-style 3D pooling module on sparse points, which is powerful and deployment-friendly without utilizing any customized CUDA operations. Our model achieves state-of-the-art performance on large-scale Waymo Open Dataset with remarkable gains. More importantly, DSVT can be easily deployed by TensorRT with realtime inference speed (27Hz). Code will be available at https://github.com/Haiyang-W/DSVT.

show abstract

Section: Detector Setupmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

Wang¹,

Chen²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…After the voxelization process, many empty grids will be generated due to the sparsity of the point cloud, which will lead to great redundant computational overheads. In order to improve computational efficiency, some methods [6,10,30,44,47] use 3D sparse convolution [14] to skip convolution calculation on empty grids. Despite being effective, sparse convolution poses a challenge when converted to ONNX/TensorRT for deployment and network quantization and hampers further speedup through these techniques.…”

Section: Introductionmentioning

confidence: 99%

“…Although PointPillars has great advantages in speed, its performance still lags far behind other methods, e.g., [47]. To boost the performance of pillar-based method, PillarNet [30] was proposed and it can achieve high-performance 3D detection performance while keeping real-time. PillarNet uses a sparse convolution-based encoder network for spatial feature learning and a neck module for high-level and low-level feature fusion.…”

Section: Introductionmentioning

confidence: 99%

FastPillars: A Deployment-friendly Pillar-based 3D Detector

Zhou¹,

Tian²,

Chu³

et al. 2023

Preprint

View full text Add to dashboard Cite

The deployment of 3D detectors strikes one of the major challenges in real world self-driving scenarios. Existing BEV-based (i.e., Bird Eye View) detectors favor sparse convolution (known as SPConv) to speed up training and inference, which puts a hard barrier for deployment especially for on-device applications. In this paper, we tackle the problem of efficient 3D object detection from LiDAR point clouds with deployment in mind. To reduce computational burden, we propose a pillar-based 3D detector with high-performance from an industry perspective, termed FastPillars. Compared with previous methods, we introduce a more effective Max-and-Attention pillar encoding (MAPE) module , and redesigning a powerful and lightweight backbone CRVNet imbued with Cross Stage Partial network (CSP) in a reparameterization style, forming a compact feature representation framework. Extensive experiments demonstrate that our FastPillars surpasses the state-of-the-art 3D detectors regarding both on-device speed and performance. Specifically, FastPillars can be effectively deployed through TensorRT, obtaining real-time performance (∼ 24FPS) on a single RTX3070Ti GPU with 64.6 mAP on the nuScenes test set. Our code is publicly available at: https://github.com/StiphyJay/ FastPillars.

show abstract

RangeLDM: Fast Realistic LiDAR Point Cloud Generation

Hu,

Zhang,

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection

Cited by 93 publications

References 40 publications

DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

FastPillars: A Deployment-friendly Pillar-based 3D Detector

RangeLDM: Fast Realistic LiDAR Point Cloud Generation

Contact Info

Product

Resources

About