RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation

Abstract

Point clouds can be represented in many forms (views), typically, point-based sets, voxel-based cells or rangebased images(i.e., panoramic view). The point-based view is geometrically accurate, but it is disordered, which makes it difficult to find local neighbors efficiently. The voxelbased view is regular, but sparse, and computation grows cubicly when voxel resolution increases. The range-based view is regular and generally dense, however spherical projection makes physical dimensions distorted. Both voxeland range-based views suffer from quantization loss, especially for voxels when facing large-scale scenes. In order to utilize different view's advantages and alleviate their own shortcomings in fine-grained segmentation task, we propose a novel range-point-voxel fusion network, namely RPVNet. In this network, we devise a deep fusion framework with multiple and mutual information interactions among these three views, and propose a gated fusion module (termed as GFM), which can adaptively merge the three features based on concurrent inputs. Moreover, the proposed RPV interaction mechanism is highly efficient, and we summarize it to a more general formulation. By leveraging this efficient interaction and relatively lower voxel resolution, our method is also proved to be more efficient. Finally, we evaluated the proposed model on two large-scale datasets, i.e., Se-manticKITTI and nuScenes, and it shows state-of-the-art performance on both of them. Note that, our method currently ranks 1st on SemanticKITTI leaderboard without any extra tricks.

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

“…Fusion-based methods (Liong et al, 2020;Xu et al, 2021;Cheng et al, 2021) use two or more of the previous methods to extract features from point clouds such as combination of voxel and point-based method, or voxel, point, projection-based method. Those methods infer the different backbones in parallel and fuse the point-wise features from each backbone using concatenation, attention, and add.…”

MVP-Net: Multiple View Pointwise Semantic Segmentation of Large-Scale Point Clouds

“…Voxelization based methods [10,11,12,13] transform the irregular unordered point cloud into regular 3D grids, and then the powerful 3D convolution is applied in feature extraction and prediction. However, the problem of granular information loss can be caused by using a large voxel size.…”

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

“…Voxels provide the coarse-grained local features, and points preserve the finegrained geometric features through a simple MLP. Xu et al [12] fuse three different feature representations, including point, range map and voxel, which achieve the promising fusion results by interacting features at various stages.…”

“…Nevertheless, they fail to preserve the original neighborhood relationship. In practice, the hybrid methods [10]- [12] fuse two or more of the above feature representations, which can obtain the better results. Unfortunately, this incurs the extra computational load.…”