Crack Detection in Paintings Using Convolutional Neural Networks

Abstract

The accurate detection of cracks in paintings, which generally portray rich and varying content, is a challenging task. Traditional crack detection methods are often lacking on recent acquisitions of paintings as they are poorly adapted to high-resolutions and do not make use of the other imaging modalities often at hand. Furthermore, many paintings portray a complex or cluttered composition, significantly complicating a precise detection of cracks when using only photographic material. In this paper, we propose a fast crack detection algorithm based on deep convolutional neural networks (CNN) that is capable of combining several imaging modalities, such as regular photographs, infrared photography and X-Ray images. Moreover, we propose an efficient solution to improve the CNN-based localization of the actual crack boundaries and extend the CNN architecture such that areas where it makes little sense to run expensive learning models are ignored. This allows us to process large resolution scans of paintings more efficiently. The proposed on-line method is capable of continuously learning from newly acquired visual data, thus further improving classification results as more data becomes available. A case study on multimodal acquisitions of the Ghent Altarpiece, taken during the currently ongoing conservation-restoration treatment, shows improvements over the state-of-the-art in crack detection methods and demonstrates the potential of our proposed method in assisting art conservators.INDEX TERMS Digital painting analysis, crack detection, virtual restoration, machine learning, morphological filtering, convolutional neural networks, transfer learning, multimodal data, Ghent Altarpiece.

Efficient Local Image Descriptors Learned With Autoencoders

“…We test the resulting method in virtual restoration of master paintings. As a case study, we use images from the panels of the Ghent Altarpiece [24], [25]. The paint-loss areas to be inpainted are detected with the algorithm from [57].…”

“…To achieve such fine-tuning with other (supervised) descriptors, it would be necessary to have a labelled set for the type of images that need to be inpainted, which is unrealistic in most cases. As a case study in this paper, we used high-resolution photographs of the panels of Ghent Altarpiece [24], [25], on which we fine-tuned the descriptor and tested our improved inpainting algorithm.…”

Adversarial Learning Approach to Unsupervised Labeling of Fine Art Paintings

“…In [32], for example, the use of DL techniques for evaluating the beauty, sentiment, and remembrance of art was explored. In [33], CNN models were applied to detect cracks in paintings.…”

A deep learning approach to crack detection on road surfaces

“…Morphological filtering is one of the most common techniques for detecting cracks. 20,21 When detecting cracks, in most cases, two main transformations are used: "Top" and "Bottom" of the hat. These transformations consist of the sequential application of two binary mathematical operations:…”