Tensor decomposition for painting analysis. Part 1: pigment characterization

Abstract

Photo-sensitive materials tend to change with exposure to light. Often, this change is visible when it affects the reflectance of the material in the visible range of the electromagnetic spectrum. In order to understand the photo-degradation mechanisms and their impact on fugitive materials, high-end scientific analysis is required. In a two-part article, we present a multi-modal approach to model fading effects in the spectral, temporal (first part) and spatial dimensions (second part). Specifically, we collect data from the same artwork, namely “A Japanese Lantern” by Norwegian artist, Oda Krohg, with two techniques, point-based microfading spectroscopy and hyperspectral imaging. In this first part, we focus on characterizing the pigments in the painting based on their spectral and fading characteristics. To begin with, using microfading data of a region in the painting, we analyze the color deterioration of the measured points. Then, we train a tensor decomposition model to reduce the measured materials to a spectral basis of unmixed pigments and, at the same time, to recover the fading rate of these endmembers (i.e. pure, unmixed chemical signals). Afterwards, we apply linear regression to predict the fading rate in the future. We validate the quality of these predictions by spectrally comparing them with temporal observations not included in the training part. Furthermore, we statistically assess the goodness of our model in explaining new data, collected from another region of the painting. Finally, we propose a visual way to explore the artist’s palette, where potential matches between endmembers and reference spectral libraries can be evaluated based on three metrics at once.

Tensor decomposition for painting analysis. Part 2: spatio-temporal simulation

“…Figure 1 displays the workflow of our approach. The first and core module is represented by the tensor decomposition model (thoroughly described in Part 1 of this twoarticle series) [10]. This model takes as input a collection of microfaded samples, and employs parallel factor analysis (PARAFAC) to find the spectra of the unmixed pigments (endmembers), their concentration in each sample and their fading rate.…”

“…For the sake of brevity, we will not insist here on the tensor decomposition model as it was described in the "Method" section of Part 1 of this article series [10]. Thus, we take for granted that matrix C represents the endmembers, A the concentration of each endmember f = {1 .…”

“…By flattening the abundance maps A map into a matrix of cardinality IJ • F , we can then replace it in Eq. 2 of Part 1 [10] and obtain spatial simulations of fading for each k time step modelled with PARAFAC:…”

“…The loadings of the 2nd mode, B define the fading rate of every endmember. In Part 1 [10], Eq. 3 we showed how the fading rate can be expressed as a linear function of the light exposure.…”

“…Our novel approach for spatio-temporal simulation of paintings is also a data-driven method. In a previous article [10], we showed how from a set of microfading measurements, we created a tensor decomposition model and extracted the spectral curve of the pure pigments, together with their temporal evolution. Now, we link the loadings of the tensor decomposition model with a hyperspectral capture of the same scene.…”