Virtual Disassembling of Historical Edifices: Experiments and Assessments of an Automatic Approach for Classifying Multi-Scalar Point Clouds into Architectural Elements

Abstract

3D heritage documentation has seen a surge in the past decade due to developments in reality-based 3D recording techniques. Several methods such as photogrammetry and laser scanning are becoming ubiquitous amongst architects, archaeologists, surveyors, and conservators. The main result of these methods is a 3D representation of the object in the form of point clouds. However, a solely geometric point cloud is often insufficient for further analysis, monitoring, and model predicting of the heritage object. The semantic annotation of point clouds remains an interesting research topic since traditionally it requires manual labeling and therefore a lot of time and resources. This paper proposes an automated pipeline to segment and classify multi-scalar point clouds in the case of heritage object. This is done in order to perform multi-level segmentation from the scale of a historical neighborhood up until that of architectural elements, specifically pillars and beams. The proposed workflow involves an algorithmic approach in the form of a toolbox which includes various functions covering the semantic segmentation of large point clouds into smaller, more manageable and semantically labeled clusters. The first part of the workflow will explain the segmentation and semantic labeling of heritage complexes into individual buildings, while a second part will discuss the use of the same toolbox to segment the resulting buildings further into architectural elements. The toolbox was tested on several historical buildings and showed promising results. The ultimate intention of the project is to help the manual point cloud labeling, especially when confronted with the large training data requirements of machine learning-based algorithms.

Connecting Geometry and Semantics via Artificial Intelligence: From 3d Classification of Heritage Data to H-Bim Representations

“…Further developments of this approach led to test the algorithm also for multi-level and multi-scale semantic segmentation (Teruggi et al, 2020). With the same goal, (Murtiyoso and Grussenmeyer, 2020) presented an algorithmic approach in the form of a toolbox that supports the manual segmentation of large point clouds, including several semi-automated pipelines.…”

A Hierarchical Machine Learning Approach for Multi-Level and Multi-Resolution 3D Point Cloud Classification

“…On the other hand, DL refers to those processes (neural networks-NN) that directly learn features and semantics from a large quantity of annotated data, which is generally not available in the heritage sector. To cope with this problem, the research proposed by [32] aims to facilitate the annotation process necessary to train DL algorithms. The authors, through a series of rule-based functions, isolate some specific architectural classes within the point cloud, such as columns and beams.…”

A Benchmark for Large-Scale Heritage Point Cloud Semantic Segmentation

“…The authors also released a dataset with more than 10k images including categories like Altar, Apse, Belltower, Column, Dome (inner and outer), Flying buttress, Gargoyle, Stained glass, and Vault. In this context, several researchers have started to approach the topic of semantic segmentation of cultural heritage (CH) point clouds within the machine and deep learning framework (Grilli et al, 2019a;Kharroubi et al, 2019;Murtiyoso and Grussenmeyer, 2020;Pierdicca et al, 2020). However, the lack of an appropriate 3D heritage dataset does not allow an effective comparison between methods and results.…”