Flex-Convolution

Many point cloud segmentation methods rely on transferring irregular points into a voxel-based regular representation. Although voxel-based convolutions are useful for feature aggregation, they produce ambiguous or wrong predictions if a voxel contains points from different classes. Other approaches (such as PointNets and point-wise convolutions) can take irregular points for feature learning. But their high memory and computational costs (such as for neighborhood search and ball-querying) limit their ability and accuracy for large-scale point cloud processing. To address these issues, we propose a deep fusion network architecture (FusionNet) with a unique voxel-based "mini-PointNet" point cloud representation and a new feature aggregation module (fusion module) for large-scale 3D semantic segmentation. Our FusionNet can learn more accurate point-wise predictions when compared to voxel-based convolutional networks. It can realize more effective feature aggregations with lower memory and computational complexity for large-scale point cloud segmentation when compared to the popular point-wise convolutions. Our experimental results show that FusionNet can take more than one million points on one GPU for training to achieve state-of-the-art accuracy on large-scale Semantic KITTI benchmark. The code will be available at https://github.com/feihuzhang/LiDARSeg.

show abstract

“…Other recent work explores the extension of convolutions on irregular and unordered point-cloud inputs [8,25,26,46,47,55].…”

Section: Point-wise Convolutionsmentioning

confidence: 99%

“…There is also plenty of work on point-based convolutions that explores the idea of convolutions directly on irregular points [8,25,26,46,47,55]. They learn to approximate a weight function or interpolate convolutional weights [10,25,28,32,50,53,55].…”

Section: Introductionmentioning

confidence: 99%

Deep FusionNet for Point Cloud Semantic Segmentation

Zhang

Jin

Wah

et al. 2020

Lecture Notes in Computer Science

130

View full text Add to dashboard Cite

show abstract

“…Recently, there is an increasing interest in designing convolutions that directly operates on point clouds, inspired by the great performance of CNN on 2D images. To design a point convolution network, the authors of [ 19 , 20 , 21 ] attempt to construct continuous kernel functions to convolve on local points. PointConv [ 19 ] uses a Multi-Layer Perceptron (MLP) to fit a kernel due to its ability to approximate an arbitrary continuous function.…”

Section: Related Workmentioning

confidence: 99%

“…SpiderCNN [ 20 ] found that the MLP did not work well on approximating the kernels, so the authors propose the order-3 Taylor term which is a family of polynomial functions applied with different weights to enrich the complexity of the filters. Flex convolution [ 21 ] utilizes linear functions to act as a kernel which is actually an order-1 Taylor term of SpiderCNN. Structure-aware Convolution (SAC) [ 22 ] matches neighbor points in the point cloud through 3D convolution to extract geometric features.…”

Section: Related Workmentioning

confidence: 99%

AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis

Yang

Zhang

et al. 2020

Sensors

View full text Add to dashboard Cite

Point set is a major type of 3D structure representation format characterized by its data availability and compactness. Most former deep learning-based point set models pay equal attention to different point set regions and channels, thus having limited ability in focusing on small regions and specific channels that are important for characterizing the object of interest. In this paper, we introduce a novel model named Attention-based Point Network (AttPNet). It uses attention mechanism for both global feature masking and channel weighting to focus on characteristic regions and channels. There are two branches in our model. The first branch calculates an attention mask for every point. The second branch uses convolution layers to abstract global features from point sets, where channel attention block is adapted to focus on important channels. Evaluations on the ModelNet40 benchmark dataset show that our model outperforms the existing best model in classification tasks by 0.7% without voting. In addition, experiments on augmented data demonstrate that our model is robust to rotational perturbations and missing points. We also design a Electron Cryo-Tomography (ECT) point cloud dataset and further demonstrate our model’s ability in dealing with fine-grained structures on the ECT dataset.

show abstract

“…Xu et al [ 7 ] processed irregular data through the parameterized filters. Groh et al [ 30 ] extended the traditional convolution to larger scale point cloud processing through exploring different parameterizations to generate the edge-dependent filters. Verma et al [ 31 ] used soft-assignment matrices to extend traditional convolution into point cloud.…”

Section: Introductionmentioning

confidence: 99%

DNet: Dynamic Neighborhood Feature Learning in Point Cloud

Tian

Jiang

2021

Sensors

View full text Add to dashboard Cite

Neighborhood selection is very important for local region feature learning in point cloud learning networks. Different neighborhood selection schemes may lead to quite different results for point cloud processing tasks. The existing point cloud learning networks mainly adopt the approach of customizing the neighborhood, without considering whether the selected neighborhood is reasonable or not. To solve this problem, this paper proposes a new point cloud learning network, denoted as Dynamic neighborhood Network (DNet), to dynamically select the neighborhood and learn the features of each point. The proposed DNet has a multi-head structure which has two important modules: the Feature Enhancement Layer (FELayer) and the masking mechanism. The FELayer enhances the manifold features of the point cloud, while the masking mechanism is used to remove the neighborhood points with low contribution. The DNet can learn the manifold features and spatial geometric features of point cloud, and obtain the relationship between each point and its effective neighborhood points through the masking mechanism, so that the dynamic neighborhood features of each point can be obtained. Experimental results on three public datasets demonstrate that compared with the state-of-the-art learning networks, the proposed DNet shows better superiority and competitiveness in point cloud processing task.

show abstract

Flex-Convolution

Cited by 77 publications

References 18 publications

Deep FusionNet for Point Cloud Semantic Segmentation

Deep FusionNet for Point Cloud Semantic Segmentation

AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis

DNet: Dynamic Neighborhood Feature Learning in Point Cloud

Contact Info

Product

Resources

About