Applying Deep Learning Techniques to Cultural Heritage Images Within the INCEPTION Project

Llamas, José; Lerones, Pedro Martín; Zalama, Eduardo; Gómez-García-Bermejo, Jaime

doi:10.1007/978-3-319-48974-2_4

Cited by 41 publications

(25 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature, different solutions are presented for the classification of architectural images, using different techniques such as pattern detection [75], Gabor filters and support vector machine [76], K-means algorithms [77], clustering and learning of local features [78], hierarchical sparse coding of blocks [79] or CNN deep learning [80,16].…”

Section: Segmentation and Classification In Cultural Heritagementioning

confidence: 99%

Classification of 3D Digital Heritage

2019

View full text Add to dashboard Cite

In recent years, the use of 3D models in cultural and archaeological heritage for documentation and dissemination purposes is increasing. The association of heterogeneous information to 3D data by means of automated segmentation and classification methods can help to characterize, describe and better interpret the object under study. Indeed, the high complexity of 3D data along with the large diversity of heritage assets themselves have constituted segmentation and classification methods as currently active research topics. Although machine learning methods brought great progress in this respect, few advances have been developed in relation to cultural heritage 3D data. Starting from the existing literature, this paper aims to develop, explore and validate reliable and efficient automated procedures for the classification of 3D data (point clouds or polygonal mesh models) of heritage scenarios. In more detail, the proposed solution works on 2D data (“texture-based” approach) or directly on the 3D data (“geometry-based approach) with supervised or unsupervised machine learning strategies. The method was applied and validated on four different archaeological/architectural scenarios. Experimental results demonstrate that the proposed approach is reliable and replicable and it is effective for restoration and documentation purposes, providing metric information e.g. of damaged areas to be restored.

show abstract

Section: Segmentation and Classification In Cultural Heritagementioning

confidence: 99%

Classification of 3D Digital Heritage

2019

View full text Add to dashboard Cite

show abstract

“…The INCEPTION project concerns the development of tools and methodologies to obtain 3D models of cultural heritage assets, enriched with semantic information and integrated on a new (H)-BIM platform. As part of the project, the work of (Llamas et al, 2016) uses deep learning techniques, in particular Convolutional Neural Networks (CNNs) for classifying images of CH. They retain that the application of these techniques can give a significant contribution to the digital documentation of cultural heritage.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning for Semantic Segmentation of 3d Point Cloud

Malinverni

Pierdicca

Paolanti

et al. 2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

<p><strong>Abstract.</strong> Cultural Heritage is a testimony of past human activity, and, as such, its objects exhibit great variety in their nature, size and complexity; from small artefacts and museum items to cultural landscapes, from historical building and ancient monuments to city centers and archaeological sites. Cultural Heritage around the globe suffers from wars, natural disasters and human negligence. The importance of digital documentation is well recognized and there is an increasing pressure to document our heritage both nationally and internationally. For this reason, the three-dimensional scanning and modeling of sites and artifacts of cultural heritage have remarkably increased in recent years. The semantic segmentation of point clouds is an essential step of the entire pipeline; in fact, it allows to decompose complex architectures in single elements, which are then enriched with meaningful information within Building Information Modelling software. Notwithstanding, this step is very time consuming and completely entrusted on the manual work of domain experts, far from being automatized. This work describes a method to label and cluster automatically a point cloud based on a supervised Deep Learning approach, using a state-of-the-art Neural Network called PointNet++. Despite other methods are known, we have choose PointNet++ as it reached significant results for classifying and segmenting 3D point clouds. PointNet++ has been tested and improved, by training the network with annotated point clouds coming from a real survey and to evaluate how performance changes according to the input training data. It can result of great interest for the research community dealing with the point cloud semantic segmentation, since it makes public a labelled dataset of CH elements for further tests.</p>

show abstract

“…Amato et al (2015) used k-nearest neighbour (kNN) classification and landmark recognition techniques to address the problem of monument recognition in images. Convolutional Neural Networks (CNN) were applied for the first time to heritage scenarios in Llamas et al (2016) and Llamas et al, (2017). CNNs are also used by Yasser et al (2017) for visual categorization and to create a digital heritage search platform (ICARE) that allows users to archive digital heritage content and perform semantic queries over multimodal cultural heritage data archives.…”

Section: Related Workmentioning

confidence: 99%

Application of Machine and Deep Learning Strategies for the Classification of Heritage Point Clouds

Grilli

Özdemir

Remondino

2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. The use of heritage point cloud for documentation and dissemination purposes is nowadays increasing. The association of semantic information to 3D data by means of automated classification methods can help to characterize, describe and better interpret the object under study. In the last decades, machine learning methods have brought significant progress to classification procedures. However, the topic of cultural heritage has not been fully explored yet. This paper presents a research for the classification of heritage point clouds using different supervised learning approaches (Machine and Deep learning ones). The classification is aimed at automatically recognizing architectural components such as columns, facades or windows in large datasets. For each case study and employed classification method, different accuracy metrics are calculated and compared.

show abstract

Applying Deep Learning Techniques to Cultural Heritage Images Within the INCEPTION Project

Cited by 41 publications

References 3 publications

Classification of 3D Digital Heritage

Classification of 3D Digital Heritage

Deep Learning for Semantic Segmentation of 3d Point Cloud

Application of Machine and Deep Learning Strategies for the Classification of Heritage Point Clouds

Contact Info

Product

Resources

About