Semantic Labelling of Ultra Dense MLS Point Clouds in Urban Road Corridors Based on Fusing CRF With Shape Priors

Abstract

ABSTRACT:In this paper, a labelling method for the semantic analysis of ultra-high point density MLS data (up to 4000 points/m 2 ) in urban road corridors is developed based on combining a conditional random field (CRF) for the context-based classification of 3D point clouds with shape priors. The CRF uses a Random Forest (RF) for generating the unary potentials of nodes and a variant of the contrastsensitive Potts model for the pair-wise potentials of node edges. The foundations of the classification are various geometric features derived by means of co-variance matrices and local accumulation map of spatial coordinates based on local neighbourhoods. Meanwhile, in order to cope with the ultra-high point density, a plane-based region growing method combined with a rule-based classifier is applied to first fix semantic labels for man-made objects. Once such kind of points that usually account for majority of entire data amount are pre-labeled; the CRF classifier can be solved by optimizing the discriminative probability for nodes within a subgraph structure excluded from pre-labeled nodes. The process can be viewed as an evidence fusion step inferring a degree of belief for point labelling from different sources. The MLS data used for this study were acquired by vehicle-borne Z+F phase-based laser scanner measurement, which permits the generation of a point cloud with an ultra-high sampling rate and accuracy. The test sites are parts of Munich City which is assumed to consist of seven object classes including impervious surfaces, tree, building roof/facade, low vegetation, vehicle and pole. The competitive classification performance can be explained by the diverse factors: e.g. the above ground height highlights the vertical dimension of houses, trees even cars, but also attributed to decision-level fusion of graph-based contextual classification approach with shape priors. The use of context-based classification methods mainly contributed to smoothing of labelling by removing outliers and the improvement in underrepresented object classes. In addition, the routine operation of a context-based classification for such high density MLS data becomes much more efficient being comparable to non-contextual classification schemes.

Mapping Individual Tree Species and Vitality along Urban Road Corridors with LiDAR and Imaging Sensors: Point Density versus View Perspective

“…Firstly, tree points were accurately separated from other type points by exploiting contextual classification with spatial semantic features. To handle ultra-dense MLS data (for example, with density larger than 1000 points/m 2 ), region growing based segmentation was adopted to extract useful planar structures, which were integrated alongside spatial context features into a conditional random field (CRF) framework for improving point labeling accuracy and efficiency [20]. Secondly, individual trees were segmented from tree points through the graph cuts segmentation method; then more than 400 features including infrared image features, geometric and intensity related features from LiDAR data were extracted for each detected tree.…”

“…In this work a context-aware classification scheme is applied based on combining constrained CRF with random forest (RF). The majority of the strategy was developed and described in [20], which is extended here to make it applicable to ALS point clouds as well. For the sake of being self-contained, in this paper we briefly review the developed approach to classify MLS point clouds and present it in a way to unify the semantic labeling for all kinds of dense point cloud data including ALS data.…”

“…Each recovered region is processed using a RANSAC (random sample consensus) plane fitting procedure. Finally, each plane is classified into one of 5 categories: car, roof, façade, ground or tree, based on the heuristic rules described in [20].…”

“…To apply the rule-based plane region classification, the probability threshold was set to 0.6 to separate tree points from original point cloud. Then based on other rules described in [20], other points were pre-labelled. After two initial labeling steps, the CRF smoothing and fusion optimization described in Section 2.2.2 was applied.…”

Semantic Labeling and Refinement of Lidar Point Clouds Using Deep Neural Network in Urban Areas

“…However, for these supervised point-wise classification, although good classification results could be produced with this straightforward procedure due to the high distinctness of hand-drafted features (Hong et al, 2015), the classification result may be in-homogeneous, especially in the areas with low point density and the boundaries of objects, due to the deficiency of the consideration of the local neighborhood of each point. To enhance the regional smoothness of the result of semantic labeling, some contextual classification methods have been proposed, such as Markov random fields (Munoz et al, 2009, Lu, Rasmussen, 2012 and conditional random fields (Niemeyer et al, 2014, Weinmann et al, 2015b, Yao et al, 2017. In this method, each point is classified considering not only the extracted features but also the features and the labels of its surrounding points.…”