Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-Based Perception

Zhu, Xinge; Zhou, Hui; Wang, Tai; Hong, Fangzhou; Li, Wei; Ma, Yuexin; Li, Hongsheng; Yang, Ruigang; Lin, Dahua

doi:10.1109/tpami.2021.3098789

Cited by 70 publications

(34 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To this end, the sparse convolution [7], [9] is employed to reduce the computational cost. Zhu et al [15] propose a voxel division method with asymmetric convolution based on LiDAR point cloud distribution.…”

Section: Related Workmentioning

confidence: 99%

“…Duerr et al [26] propose a recurrent segmentation architecture using range images, which recursively aggregate the features of previous scans in order to exploit the short term temporal dependencies. In [15], superimposing point clouds in 3D space is adopted for multiple scans segmentation, whose memory consumption and computational time increase linearly with the total number of scans per input model. In this paper, we introduce an efficient range residual image representation, where the effective features can be extracted by Meta-Kernel.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, we aim to predicting the semantic labels from the consecutive LiDAR sequences. Unlike the conventional approaches transforming the sequential point cloud into global coordinates [14], [15], we suggest a novel Meta-RangeSeg approach to efficiently process multiple scans in range view for the subsequent feature extraction.…”

Section: A Overviewmentioning

confidence: 99%

“…Most of previous approaches only take into account of single frame in LiDAR semantic segmentation, where the important temporal information is usually ignored. Moreover, some methods [14], [15] aim to deal with multiple scans simultaneously. This may lead to information redundancy and slow inference speed, as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

Wang¹,

Zhu²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

LiDAR sensor is essential to the perception system in autonomous vehicles and intelligent robots. To fulfill the realtime requirements in real-world applications, it is necessary to efficiently segment the LiDAR scans. Most of previous approaches directly project 3D point cloud onto the 2D spherical range image so that they can make use of the efficient 2D convolutional operations for image segmentation. Although having achieved the encouraging results, the neighborhood information is not wellpreserved in the spherical projection. Moreover, the temporal information is not taken into consideration in the single scan segmentation task. To tackle these problems, we propose a novel approach to semantic segmentation for LiDAR sequences named Meta-RangeSeg, where a novel range residual image representation is introduced to capture the spatial-temporal information. Specifically, Meta-Kernel is employed to extract the meta features, which reduces the inconsistency between the 2D range image coordinates input and Cartesian coordinates output. An efficient U-Net backbone is used to obtain the multi-scale features. Furthermore, Feature Aggregation Module (FAM) aggregates the meta features and multi-scale features, which tends to strengthen the role of range channel. We have conducted extensive experiments for performance evaluation on SemanticKITTI, which is the de-facto dataset for LiDAR semantic segmentation. The promising results show that our proposed Meta-RangeSeg method is more efficient and effective than the existing approaches. Our full implementation is publicly available at https://github.com/songw-zju/Meta-RangeSeg.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

Wang¹,

Zhu²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…MinkowskiNet [39] utilizes the sparse convolutions to efficiently perform semantic segmentation on the voxelized large-scale point clouds. SqueezeSeg [5] views LiDAR point clouds as range images while PolarNet [6] and Cylinder3D [11], [40], [41] divide the LiDAR point clouds under the polar and cylindrical coordinate systems.…”

Section: Related Workmentioning

confidence: 99%

LiDAR-based Panoptic Segmentation via Dynamic Shifting Network

Hong

Zhou²,

Zhu

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

With the rapid advances of autonomous driving, it becomes critical to equip its sensing system with more holistic 3D perception. However, existing works focus on parsing either the objects (e.g. cars and pedestrians) or scenes (e.g. trees and buildings) from the LiDAR sensor. In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner. As one of the first endeavors towards this new challenging task, we propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. In particular, DS-Net has three appealing properties: 1) Strong backbone design. DS-Net adopts the cylinder convolution that is specifically designed for LiDAR point clouds. The extracted features are shared by the semantic branch and the instance branch which operates in a bottom-up clustering style. 2) Dynamic Shifting for complex point distributions. We observe that commonly-used clustering algorithms like BFS or DBSCAN are incapable of handling complex autonomous driving scenes with non-uniform point cloud distributions and varying instance sizes. Thus, we present an efficient learnable clustering module, dynamic shifting, which adapts kernel functions on the fly for different instances. 3) Extension to 4D prediction. Furthermore, we extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames. To comprehensively evaluate the performance of LiDAR-based panoptic segmentation, we construct and curate benchmarks from two large-scale autonomous driving LiDAR datasets, SemanticKITTI and nuScenes. Extensive experiments demonstrate that our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks. Notably, in the single frame version of the task, we outperform the SOTA method by 1.8% in terms of the PQ metric. In the 4D version of the task, we surpass 2nd place by 5.4% in terms of the LSTQ metric.

show abstract

PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection

Shi

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-Based Perception

Cited by 70 publications

References 61 publications

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

LiDAR-based Panoptic Segmentation via Dynamic Shifting Network

PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection

Contact Info

Product

Resources

About