Point cloud semantic scene segmentation based on coordinate convolution

Abstract

Point cloud semantic segmentation, a crucial research area in the 3D computer vision, lies at the core of many vision and robotics applications. Due to the irregular and disordered of the point cloud, however, the application of convolution on point clouds is challenging. In this article, we propose the “coordinate convolution,” which can effectively extract local structural information of the point cloud, to solve the inapplicability of conventional convolution neural network (CNN) structures on the 3D point cloud. The “coordinate convolution” is a projection operation of three planes based on the local coordinate system of each point. Specifically, we project the point cloud on three planes in the local coordinate system with a joint 2D convolution operation to extract its features. Additionally, we leverage a self‐encoding network based on image semantic segmentation U‐Net structure as the overall architecture of the point cloud semantic segmentation algorithm. The results demonstrate that the proposed method exhibited excellent performances for point cloud data sets corresponding to various scenes.

MLFNet- Point Cloud Semantic Segmentation Convolution Network Based on Multi-Scale Feature Fusion

“…The coordinates after processing are P and P ∈ R N×M×3 ; therefore,(F ⊕ P) ∈ R N×M×(l+3) . The format of space accords with the method proposed by [3]. As with that method, when there is no near point around a point, F is represented by P; that is, the coordinate information of the obtained point is used as feature information.…”

“…The features of some point clouds cannot be accurately conveyed and expressed by logistic regression. Spatial and geometric features are ignored.In order to solve the irregularity and disorder of the point cloud, coordinate convolution [3] can effectively extract local information. The point cloud surface structure is damaged by the downsampling of the voxel grid.Since the advanced geometric correlation between the input and its neighboring coordinates and features is not fully utilized,suboptimal segmentation performance is achieved [4].There are errors in segmentation of multi-connected regions.The input of the network should be normalized to the specified number of points,information is lost by the sampling process [5].…”

DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion

“…3D vision has drawn increasing attention recently with the rapid development of 3D sensing technologies. It brings out many challenging 3D tasks, such as point cloud recognition( [16,24,46]), shape [52] and scene [27,51,53] segmentation, object detection based on point cloud [7,29,30,38,56] and monocular image [20,42,47], point cloud registration [1,40,50]. Unlike 2D images that consist of pixels in uniform grids, a 3D point cloud is permutation invariant, spatially irregular, and density varying, which leads to Figure 1.…”