Interactive Capture and Labelling of Point Clouds With Hololens2 for Semantic Segmentation

Abstract

Abstract. Low-cost capture systems allow faster data acquisition compared to other systems, although obtained resolution is lower. These systems can be used to acquire low-quality point clouds to classify with simple segmentation algorithms. In this work, we use Microsoft HoloLens 2 sensors to scan indoor 3D environments and hand tracking to position label tags that will support the point cloud segmentation based on heuristic algorithms. Thus, from each tag, several algorithms have been designed to segment doors, windows, columns, walls, ceilings, and floors. The method was tested on 3 real case studies, obtaining F1-Score between 0.61 and 0.96 depending on the object class. These results reinforce the idea that, by combining Mixed Reality with basic point cloud processing algorithms, low-quality point cloud data can be correctly processed without resorting to complex Artificial Intelligence techniques and without labelling large amounts of samples.