Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters

Abstract

In this paper, we present an efficient semantic segmentation framework for indoor scenes operating on 3D point clouds. We use the results of a Random Forest Classifier to initialize the unary potentials of a densely interconnected Conditional Random Field, for which we learn the parameters for the pairwise potentials from training data. These potentials capture and model common spatial relations between class labels, which can often be observed in indoor scenes. We evaluate our approach on the popular NYU Depth datasets, for which it achieves superior results compared to the current state of the art. Exploiting parallelization and applying an efficient CRF inference method based on mean field approximation, our framework is able to process full resolution Kinect point clouds in half a second on a regular laptop, more than twice as fast as comparable methods.

Semantic segmentation priors for object discovery

“…The dataset contains 795 training and 654 testing samples. The training set is used to learn the network parameters as explained in Section III-B and 3 Available at http://cs.nyu.edu/ ∼ silberman/datasets/nyu depth v2.html Couprie et al [29] 64.5 63.5 Khan et al [30] 69.2 65.6 Stückler et al [31] 70.9 67.0 Müller and Behnke [32] 72.3 71.9 Wolf et al [33] 72.6 74.1 Eigen and Fergus [17] 79.1 80.6 Husain et al [4] …”

Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

“…RELATED WORK The conventional approach to semantic labeling is carried out in multiple stages [4,[15][16][17][18][19]. This involves presegmenting the scene into smaller patches followed by feature extraction and classification.…”

“…Hermans et al [17] used a randomized decision forest for semantic segmentation, where the results were further refined using a dense CRF. Similarly, segmentation followed by a random forest classification to initialize the unary potentials of a CRF was proposed by Wolf et al [18].…”

RGB-D Segmentation of Poultry Entrails

“…In this paper we apply a similar framework to objects with significantly different characteristics. Our system differs from [15], by not including the CRF that is used for refining the labeling and by omitting the features that are specific to man-made object, as well as the features that utilize the orientation of the room. Figure 2 gives an overview of the pipeline for segmenting entrails into heart, liver, lung and misc.…”

“…Two widely used datasets with these types of objects are the NYU V1 [12] and V2 [8] datasets. [15] is an example of recent work that addresses this type of data. They segment the point cloud into supervoxels and use a Random Forest (RF) classifier to initialize the unary potentials of a densely interconnected Conditional Random Field (CRF).…”