Artificial intelligence for art investigation: Meeting the challenge of separating x-ray images of the
            <i>Ghent Altarpiece</i>

Sabetsarvestani, Zahra; Sober, Barak; Higgitt, Catherine; Daubechies, Ingrid; Rodrigues, Miguel R. D.

doi:10.1126/sciadv.aaw7416

Cited by 31 publications

(31 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the particular application of technical examination of art, some approaches have been proposed to separate mixed X-ray images with double-sided paintings, where RGB images pertaining to both front and back sides of the art work are available. These include sparsity-based methods [14], Gaussian mixture modelbased approaches [15] and deep neural networks [16]- [18].…”

Section: Related Workmentioning

confidence: 99%

Mixed X-Ray Image Separation for Artworks with Concealed Designs

Pu,

Huang,

Sober

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we focus on X-ray images of paintings with concealed sub-surface designs (e.g., deriving from reuse of the painting support or revision of a composition by the artist), which include contributions from both the surface painting and the concealed features. In particular, we propose a self-supervised deep learning-based image separation approach that can be applied to the X-ray images from such paintings to separate them into two hypothetical X-ray images. One of these reconstructed images is related to the X-ray image of the concealed painting, while the second one contains only information related to the Xray of the visible painting. The proposed separation network consists of two components: the analysis and the synthesis sub-networks. The analysis sub-network is based on learned coupled iterative shrinkage thresholding algorithms (LCISTA) designed using algorithm unrolling techniques, and the synthesis sub-network consists of several linear mappings. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The proposed method is demonstrated on a real painting with concealed content, Do ña Isabel de Porcel by Francisco de Goya, to show its effectiveness.

show abstract

Section: Related Workmentioning

confidence: 99%

Mixed X-Ray Image Separation for Artworks with Concealed Designs

Pu,

Huang,

Sober

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, deep convolutional neural networks (CNNs) have provided superior performance in computer vision [1], [2] and natural language processing [3]. The superior accuracy of CNNs, however, comes at a high computational complexity and energy cost.…”

Section: Introductionmentioning

confidence: 99%

Similarity-Aware CNN for Efficient Video Recognition at the Edge

Sabet

Hare

Al-Hashimi

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) often extract similar features from successive video frames due to having identical appearances. In contrast, conventional CNNs for video recognition process individual frames with a fixed computational effort. Each video frame is independently processed, resulting in numerous redundant computations and an inefficient use of limited energy resources, particularly for edge computing applications. To alleviate the high energy requirements associated with video frame processing, this paper presented similarityaware CNNs that recognise similar feature pixels across frames and avoid computations on them. First, with a loss of less than 1% in recognition accuracy, a proposed similarity aware quantization technique increases the average number of unchanged feature pixels across frame pairs by up to 85%. Then, a proposed similarity-aware dataflow improves energy consumption by minimising redundant computations and memory accesses across frame pairs. According to simulation experiments, the proposed dataflow decreases the energy consumed by video frame processing by up to 30%.

show abstract

“…Labeling of the paint classes into their component pigments (i.e., labeled pigment maps) is done either by identifying characteristic reflectance spectral features or by spatially fusing the class maps with results from other analytical methods, e.g., XRF, extended-range reflectance (near-ultraviolet, near-infrared, and mid-infrared), and Raman spectroscopies, which provide more detailed chemical information. One promising non-linear unmixing model uses a convolution neural network architecture to separate X-ray images of artwork painted on both sides of their support [26].…”

Section: Introductionmentioning

confidence: 99%

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

et al. 2020

View full text Add to dashboard Cite

Spectral imaging modalities, including reflectance and X-ray fluorescence, play an important role in conservation science. In reflectance hyperspectral imaging, the data are classified into areas having similar spectra and turned into labeled pigment maps using spectral features and fusing with other information. Direct classification and labeling remain challenging because many paints are intimate pigment mixtures that require a non-linear unmixing model for a robust solution. Neural networks have been successful in modeling non-linear mixtures in remote sensing with large training datasets. For paintings, however, existing spectral databases are small and do not encompass the diversity encountered. Given that painting practices are relatively consistent within schools of artistic practices, we tested the suitability of using reflectance spectra from a subgroup of well-characterized paintings to build a large database to train a one-dimensional (spectral) convolutional neural network. The labeled pigment maps produced were found to be robust within similar styles of paintings.

show abstract

Artificial intelligence for art investigation: Meeting the challenge of separating x-ray images of the Ghent Altarpiece

Cited by 31 publications

References 30 publications

Mixed X-Ray Image Separation for Artworks with Concealed Designs

Mixed X-Ray Image Separation for Artworks with Concealed Designs

Similarity-Aware CNN for Efficient Video Recognition at the Edge

An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence

Contact Info

Product

Resources

About