3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation

Alonso, Íñigo; Riazuelo, Luis; Montesano, Luis; Murillo, Ana C.

doi:10.48550/arxiv.2002.10893

Cited by 5 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, spherical projection provides an efficient way to sample points and the projected images preserve geometric information of point cloud well and can be processed by standard CNNs effectively. A series of methods adopted this approach [45,14,15,16,46,47,48].…”

Section: Deep Learning Methodsmentioning

confidence: 99%

“…Following prior work by [14], existing methods [14,15,46,16,48,47] stack coordinates (x,y,z), depths and intensities as five channels in the projected images, i.e.…”

Section: Modality Gap and Proposed Solutionmentioning

confidence: 99%

“…The widely-used spherical projection [14,15,16,46,48,47,17] transfers 3D point cloud into 2D images through following equation Φ. The pixel coordinate (u, v) of a point with spatial coordinate (x, y, z) in a single scan of point cloud could be calculated as:…”

Section: Mapping From 3d To 2dmentioning

confidence: 99%

See 2 more Smart Citations

FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation

Xiao,

Yang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Scene understanding based on LiDAR point cloud is an essential task for autonomous cars to drive safely, which often employs spherical projection to map 3D point cloud into multi-channel 2D images for semantic segmentation. Most existing methods simply stack different point attributes/modalities (e.g. coordinates, intensity, depth, etc.) as image channels to increase information capacity, but ignore distinct characteristics of point attributes in different image channels. We design FPS-Net, a convolutional fusion network that exploits the uniqueness and discrepancy among the projected image channels for optimal point cloud segmentation. FPS-Net adopts an encoder-decoder structure. Instead of simply stacking multiple channel images as a single input, we group them into different modalities to first learn modality-specific features separately and then map the learnt features into a common high-dimensional feature space for pixel-level fusion and learning. Specifically, we design a residual dense block with multiple receptive fields as a building block in encoder which preserves detailed information in each modality and learns hierarchical modalityspecific and fused features effectively. In the FPS-Net decoder, we use a recurrent convolution block likewise to hierarchically decode fused features into output space for pixel-level classification. Extensive experiments conducted on two widely adopted point cloud datasets show that FPS-Net achieves superior semantic segmentation as compared with state-of-the-art projection-based methods. In addition, the proposed modality fusion idea is compatible with typical projection-based methods and can be incorporated into them with consistent performance improvement.

show abstract

Section: Deep Learning Methodsmentioning

confidence: 99%

“…Following prior work by [14], existing methods [14,15,46,16,48,47] stack coordinates (x,y,z), depths and intensities as five channels in the projected images, i.e.…”

Section: Modality Gap and Proposed Solutionmentioning

confidence: 99%

See 1 more Smart Citation

FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation

Xiao,

Yang,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Considering the imbalanced spatial distribution of the LiDAR point clouds, PolarNet [21] voxelizes raw point cloud in a BEV polar grid, and the points in each grid is produced by a learnable simplified PointNet. 3D-MiniNet [22] groups points into view frustums. Point-wise method is implemented in each view frustum, and the global context information is mainly extracted with the MiniNet backbone, which is a 2D CNN operating on range image.…”

Section: Related Workmentioning

confidence: 99%

S3Net: 3D LiDAR Sparse Semantic Segmentation Network

Cheng

Razani

Ren

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Semantic Segmentation is a crucial component in the perception systems of many applications, such as robotics and autonomous driving that rely on accurate environmental perception and understanding. In literature, several approaches are introduced to attempt LiDAR semantic segmentation task, such as projection-based (range-view or birds-eye-view), and voxel-based approaches. However, they either abandon the valuable 3D topology and geometric relations and suffer from information loss introduced in the projection process or are inefficient. Therefore, there is a need for accurate models capable of processing the 3D driving-scene point cloud in 3D space. In this paper, we propose S3Net, a novel convolutional neural network for LiDAR point cloud semantic segmentation. It adopts an encoder-decoder backbone that consists of Sparse Intra-channel Attention Module (SIntraAM), and Sparse Interchannel Attention Module (SInterAM) to emphasize the fine details of both within each feature map and among nearby feature maps. To extract the global contexts in deeper layers, we introduce Sparse Residual Tower based upon sparse convolution that suits varying sparsity of LiDAR point cloud. In addition, geo-aware anisotropic loss is leveraged to emphasize the semantic boundaries and penalize the noise within each predicted regions, leading to a robust prediction. Our experimental results show that the proposed method leads to a large improvement (12%) compared to its baseline counterpart (MinkNet42 [1]) on SemanticKITTI [2] test set and achieves state-of-the-art mIoU accuracy of semantic segmentation approaches.

show abstract

“…The contributions of this work mainly lie in three aspects: (1) We reposition the focus of outdoor LiDAR segmentation from 2D projection to 3D structure, and further investigate the inherent properties (difficulties) of outdoor point cloud. (2) We introduce a new framework to explore the 3D geometric pattern and tackle these difficulties caused by sparsity and varying density, through cylindrical partition and asymmetrical 3D convolution networks.…”

Section: Introductionmentioning

confidence: 99%

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Zhu¹,

Zhou²,

Wang³

et al. 2020

Preprint

View full text Add to dashboard Cite

State-of-the-art methods for large-scale driving-scene LiDAR segmentation often project the point clouds to 2D space and then process them via 2D convolution. Although this corporation shows the competitiveness in the point cloud, it inevitably alters and abandons the 3D topology and geometric relations. A natural remedy is to utilize the 3D voxelization and 3D convolution network. However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited. An important reason is the property of the outdoor point cloud, namely sparsity and varying density. Motivated by this investigation, we propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern while maintaining these inherent properties. Moreover, a point-wise refinement module is introduced to alleviate the interference of lossy voxel-based label encoding. We evaluate the proposed model on two large-scale datasets, i.e., SemanticKITTI and nuScenes. Our method achieves the 1st place in the leaderboard of SemanticKITTI 1 and outperforms existing methods on nuScenes with a noticeable margin, about 4%. Furthermore, the proposed 3D framework also generalizes well to LiDAR panoptic segmentation and LiDAR 3D detection.

show abstract

3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation

Cited by 5 publications

References 34 publications

FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation

FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation

S3Net: 3D LiDAR Sparse Semantic Segmentation Network

Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

Contact Info

Product

Resources

About