VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention

Deng, Shengheng; Liang, Zhihao; Sun, Lin; Jia, Kui

doi:10.1109/cvpr52688.2022.00826

Cited by 64 publications

(15 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that both FastPillars and PillarNet make use of the same training settings in the experiment. Particularly, compared with VISTA [7], the mAP/NDS of our FastPillars-s/FastPillars-m outperforms it 1.6%/2.8% and 0.3%/1.2% respectively. The performance of FastPillarss and FastPillars-m could be further boosted by a substantial margin 1.4%/1.0% in mAP/NDS (from 64.6% to 66.0%/70.1% to 71.1% ) and 1.0%/0.8% (from 65.8% to 66.8%/71.0% to 71.8%) with the fade strategy, respectively.…”

Section: Methodsmentioning

confidence: 86%

FastPillars: A Deployment-friendly Pillar-based 3D Detector

Zhou¹,

Tian²,

Chu³

et al. 2023

Preprint

View full text Add to dashboard Cite

The deployment of 3D detectors strikes one of the major challenges in real world self-driving scenarios. Existing BEV-based (i.e., Bird Eye View) detectors favor sparse convolution (known as SPConv) to speed up training and inference, which puts a hard barrier for deployment especially for on-device applications. In this paper, we tackle the problem of efficient 3D object detection from LiDAR point clouds with deployment in mind. To reduce computational burden, we propose a pillar-based 3D detector with high-performance from an industry perspective, termed FastPillars. Compared with previous methods, we introduce a more effective Max-and-Attention pillar encoding (MAPE) module , and redesigning a powerful and lightweight backbone CRVNet imbued with Cross Stage Partial network (CSP) in a reparameterization style, forming a compact feature representation framework. Extensive experiments demonstrate that our FastPillars surpasses the state-of-the-art 3D detectors regarding both on-device speed and performance. Specifically, FastPillars can be effectively deployed through TensorRT, obtaining real-time performance (∼ 24FPS) on a single RTX3070Ti GPU with 64.6 mAP on the nuScenes test set. Our code is publicly available at: https://github.com/StiphyJay/ FastPillars.

show abstract

Section: Methodsmentioning

confidence: 86%

FastPillars: A Deployment-friendly Pillar-based 3D Detector

Zhou¹,

Tian²,

Chu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The voxel-based approaches [1], [2], [7], [13], [15], [16], [20], [21] initially split point clouds into 3D voxels through voxelization, owing to the unstructured nature of point clouds and the varying point density. To be more specific, the voxelization process is a maximal or average pooling operation of the points in each voxel, thus producing voxel features.…”

Section: A Voxel-based Methodsmentioning

confidence: 99%

“…To achieve efficiency, most current advanced approaches propose to utilize the gridbased representation and execute 3D object detection in the BEV space. Grid-based methods necessitate a voxelization process to transform unstructured point clouds into 3D voxels or 2D pillars, thus can be separated into voxel-based [1], [2], [7], [13]- [16], [20], [21], [23] and pillar-based [3], [8]- [10], [18], [22] approaches. After voxelization, the voxelbased approaches employ a 3D voxel backbone to encode voxel features, which are then flattened along the vertical axis to acquire 2D BEV features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

et al. 2023

View full text Add to dashboard Cite

Point cloud object detection is a pivotal technology in autonomous driving and robotics. Currently, the majority of cutting-edge point cloud detectors utilize Bird's Eye View (BEV) for detection, as it allows them to take advantage of well-explored 2D detection techniques. Nevertheless, dimensionality reduction of features from 3D space to BEV space unavoidably leads to information loss, and there is a lack of research on this issue. Existing methods typically obtain BEV features by collapsing voxel or point features along the height dimension via a pooling operation or convolution, resulting in a significant decrease in geometric information. To tackle this problem, we present a new point cloud backbone network for grid-based object detection, MDRNet, which is based on adaptive dimensionality reduction and multilevel spatial residual strategies. In MDRNet, the Spatial-aware Dimensionality Reduction (SDR) is designed to dynamically concentrate on the essential components of the object during 3D-to-BEV transformation. Moreover, the Multi-level Spatial Residuals (MSR) strategy is proposed to effectively fuse multi-level spatial information in BEV feature maps. Our MDRNet can be employed on any existing grid-based object detector, resulting in a remarkable improvement in performance. Numerous experiments conducted on nuScenes, KITTI and DAIR-V have shown that MDRNet surpasses existing SOTA approaches. In particular, on the nuScenes dataset, we attained an impressive 7.2% mAP and 5.0% NDS enhancement compared with CenterPoint.

show abstract

“…6D object pose estimation is a fundamental task of 3D semantic analysis with many real-world applications, such as robotic grasping [7,44], augmented reality [27], and autonomous driving [8,9,21,42]. Non-linearity of the rotation space of SO(3) makes it hard to handle this nontrivial task through direct pose regression from object observations [6, 11, 15, 18, 24-26, 39, 45, 47].…”

Section: Introductionmentioning

confidence: 99%

DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation

Li¹,

Lin²,

Jia³

2022

Preprint

View full text Add to dashboard Cite

Establishment of point correspondence between camera and object coordinate systems is a promising way to solve 6D object poses. However, surrogate objectives of correspondence learning in 3D space are a step away from the true ones of object pose estimation, making the learning suboptimal for the end task. In this paper, we address this shortcoming by introducing a new method of Deep Correspondence Learning Network for direct 6D object pose estimation, shortened as DCL-Net. Specifically, DCL-Net employs dual newly proposed Feature Disengagement and Alignment (FDA) modules to establish, in the feature space, partial-to-partial correspondence and complete-to-complete one for partial object observation and its complete CAD model, respectively, which result in aggregated pose and match feature pairs from two coordinate systems; these two FDA modules thus bring complementary advantages. The match feature pairs are used to learn confidence scores for measuring the qualities of deep correspondence, while the pose feature pairs are weighted by confidence scores for direct object pose regression. A confidence-based pose refinement network is also proposed to further improve pose precision in an iterative manner. Extensive experiments show that DCL-Net outperforms existing methods on three benchmarking datasets, including YCB-Video, LineMOD, and Oclussion-LineMOD; ablation studies also confirm the efficacy of our novel designs. Our code is released publicly at https://github.com/Gorilla-Lab-SCUT/DCL-Net.

show abstract

VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention

Cited by 64 publications

References 15 publications

FastPillars: A Deployment-friendly Pillar-based 3D Detector

FastPillars: A Deployment-friendly Pillar-based 3D Detector

Enhancing Grid-Based 3D Object Detection in Autonomous Driving With Improved Dimensionality Reduction

DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation

Contact Info

Product

Resources

About