Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification

Abstract

Point cloud classification plays an important role in a wide range of airborne light detection and ranging (LiDAR) applications, such as topographic mapping, forest monitoring, power line detection, and road detection. However, due to the sensor noise, high redundancy, incompleteness, and complexity of airborne LiDAR systems, point cloud classification is challenging. Traditional point cloud classification methods mostly focus on the development of handcrafted point geometry features and employ machine learning-based classification models to conduct point classification. In recent years, the advances of deep learning models have caused researchers to shift their focus towards machine learning-based models, specifically deep neural networks, to classify airborne LiDAR point clouds. These learning-based methods start by transforming the unstructured 3D point sets to regular 2D representations, such as collections of feature images, and then employ a 2D CNN for point classification. Moreover, these methods usually need to calculate additional local geometry features, such as planarity, sphericity and roughness, to make use of the local structural information in the original 3D space. Nonetheless, the 3D to 2D conversion results in information loss. In this paper, we proposed a directionally constrained fully convolutional neural network (D-FCN) that can take the original 3D coordinates and LiDAR intensity as input; thus, it can directly apply to unstructured 3D point clouds for semantic labeling. Specifically, we first introduce a novel directionally constrained point convolution (D-Conv) module to extract locally representative features of 3D point sets from the projected 2D receptive fields. To make full use of the orientation information of neighborhood points, the proposed D-Conv module performs convolution in an orientation-aware manner by using a directionally constrained nearest neighborhood search. Then, we designed a multiscale fully convolutional neural network with downsampling and upsampling blocks to enable multiscale point feature learning. The proposed D-FCN model can therefore process input point cloud with arbitrary sizes and directly predict the semantic labels for all the input points in an end-to-end manner. Without involving additional geometry features as input, the proposed method has demonstrated superior performance on the International Society for Photogrammetry and Remote Sensing (ISPRS) 3D labeling benchmark dataset. The results show that our model has achieved a new state-of-the-art level of performance with an average F1 score of 70.7%, and it has improved the performance by a large margin on categories with a small number of points (such as powerline, car, and facade).

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

“…To address these problems of the information lost in the translation process, a CNN-like network called PointNet was proposed to handle 3D point clouds. Additionally, some studies have applied CNN based techniques to irregular point clouds [ 37 , 38 , 39 , 40 , 41 , 42 ] after PointNet was proposed. These methods offer an integrated architecture that avoids high computational costs coming with high resolution voxels and allows point cloud data to be entered directly for semantic segmentation tasks.…”

A Graph-Voxel Joint Convolution Neural Network for ALS Point Cloud Segmentation

“…The framework achieves the highest overall accuracy among the existing methods. Wen C et al [49] proposed a directionally constrained point convolution (D-Conv) module and a directionally constrained fully convolutional neural network (D-FCN) to directly process large-scale point clouds.…”

Semantic Segmentation for Full-Waveform LiDAR Data Using Local and Hierarchical Global Feature Extraction

“…Finally, another MLP layer was used to classify each layer. Wen et al [41] proposed a multiscale FCN that considered direction. Winiwarter et al [42] investigated the applicability of PointNet++ for semantic classification of not only benchmark data but also actual airborne LiDAR point clouds.…”

“…In this case, when point clouds are converted to voxels, classification performance is adversely affected because of information loss. However, some studies have applied CNN techniques on irregular point clouds [39,41,44,49]. These methods have shown state-of-the-art performance on several point cloud semantic segmentation benchmarks.…”