SEMANTIC3D.NET: A New Large-Scale Point Cloud Classification Benchmark

Abstract

ABSTRACT:This paper presents a new 3D point cloud classification benchmark data set with over four billion manually labelled points, meant as input for data-hungry (deep) learning methods. We also discuss first submissions to the benchmark that use deep convolutional neural networks (CNNs) as a work horse, which already show remarkable performance improvements over state-of-the-art. CNNs have become the de-facto standard for many tasks in computer vision and machine learning like semantic segmentation or object detection in images, but have no yet led to a true breakthrough for 3D point cloud labelling tasks due to lack of training data. With the massive data set presented in this paper, we aim at closing this data gap to help unleash the full potential of deep learning methods for 3D labelling tasks. Our semantic3D.net data set consists of dense point clouds acquired with static terrestrial laser scanners. It contains 8 semantic classes and covers a wide range of urban outdoor scenes: churches, streets, railroad tracks, squares, villages, soccer fields and castles. We describe our labelling interface and show that our data set provides more dense and complete point clouds with much higher overall number of labelled points compared to those already available to the research community. We further provide baseline method descriptions and comparison between methods submitted to our online system. We hope semantic3D.net will pave the way for deep learning methods in 3D point cloud labelling to learn richer, more general 3D representations, and first submissions after only a few months indicate that this might indeed be the case.

Revue des descripteurs tridimensionnels (3D) pour la catégorisation des nuages de points acquis avec un système LiDAR de télémétrie mobile

“…La préparation de ces jeux de données représente plusieurs difficultés compte tenu de la grande superficie à couvrir et donc de la quantité massive de points contenus dans les fichiers, du nombre important d'instances de classes à inclure dans les échantillons d'apprentissage et de tests, de la nécessité de labelliser et de segmenter les points souvent effectué de manière manuelle. Malgré ces difficultés, plusieurs initiatives ont déjà été lancées en ce sens (Paparoditis et al 2014;Serna et al 2014;Hackel et al 2017).…”

Integrating Deep Semantic Segmentation Into 3-D Point Cloud Registration

“…In this work, we introduce the use of real semantics, resulting from a classifier, by using PointNet [5], a deep-learning framework to learn semantic labels from data. In this paper, we examine the registration procedure by using the manually labeled data set Semantic3d.net [6].…”

“…We use the Semantic3d.net [6], a large-scale point cloud classification benchmark. The data set contains 30 labeled scans, from rural and urban scenes, in total of 4 billion points.…”

“…We train the semantic classifier on the manually labeled data set Semantic3d.net [6]. We also present the conceptual equivalent of our algorithm for GICP, referred to as SE-GICP.…”

Generation of Ground Truth Datasets for the Analysis of 3d Point Clouds in Urban Scenes Acquired via Different Sensors

“…To conduct such experiments and evaluations, the benchmark dataset or the ground truth are normally required. In fact, lots of efforts have been paid to the generation of the benchmark dataset in the community of point clouds processing, for example, the ISPRS Benchmark Test on Urban Object Detection and Reconstruction, containing challenging aerial laser scanning point clouds for 3D object reconstruction , semantic 3D benchmark for classification having large-scale terrestrial point clouds of various urban, suburban, and rural scene (Hackel et al, 2017).…”