Hyper-Spectral Imaging Technique in the Cultural Heritage Field: New Possible Scenarios

Spectral reconstruction (SR) algorithms attempt to recover hyperspectral information from RGB camera responses. Recently, the most common metric for evaluating the performance of SR algorithms is the Mean Relative Absolute Error (MRAE)—an ℓ1 relative error (also known as percentage error). Unsurprisingly, the leading algorithms based on Deep Neural Networks (DNN) are trained and tested using the MRAE metric. In contrast, the much simpler regression-based methods (which actually can work tolerably well) are trained to optimize a generic Root Mean Square Error (RMSE) and then tested in MRAE. Another issue with the regression methods is—because in SR the linear systems are large and ill-posed—that they are necessarily solved using regularization. However, hitherto the regularization has been applied at a spectrum level, whereas in MRAE the errors are measured per wavelength (i.e., per spectral channel) and then averaged. The two aims of this paper are, first, to reformulate the simple regressions so that they minimize a relative error metric in training—we formulate both ℓ2 and ℓ1 relative error variants where the latter is MRAE—and, second, we adopt a per-channel regularization strategy. Together, our modifications to how the regressions are formulated and solved leads to up to a 14% increment in mean performance and up to 17% in worst-case performance (measured with MRAE). Importantly, our best result narrows the gap between the regression approaches and the leading DNN model to around 8% in mean accuracy.

Section: Introductionmentioning

confidence: 99%

On the Optimization of Regression-Based Spectral Reconstruction

Lin

Finlayson

2021

“…The Italian paintings were acquired using a line scanning over the visible and infrared spectrum, but the spectral and spatial resolution of the data used was adjusted to be coincident with the data of the other images [32][33][34]. To achieve so, a spatial sampling was done to reduce the spatial resolution and the spectral resolution was trimmed and sampled from 400 nm to 720 nm in 10 nm steps.…”

Section: Paintings and Natural Scenesmentioning

confidence: 99%

How Good Are RGB Cameras Retrieving Colors of Natural Scenes and Paintings?—A Study Based on Hyperspectral Imaging

Linhares

Monteiro

Bailão

et al. 2020

Self Cite

RGB digital cameras (RGB) compress the spectral information into a trichromatic system capable of approximately representing the actual colors of objects. Although RGB digital cameras follow the same compression philosophy as the human eye (OBS), the spectral sensitivity is different. To what extent they provide the same chromatic experiences is still an open question, especially with complex images. We addressed this question by comparing the actual colors derived from spectral imaging with those obtained with RGB cameras. The data from hyperspectral imaging of 50 natural scenes and 89 paintings was used to estimate the chromatic differences between OBS and RGB. The corresponding color errors were estimated and analyzed in the color spaces CIELAB (using the color difference formulas ΔE*ab and CIEDE2000), Jzazbz, and iCAM06. In CIELAB the most frequent error (using ΔE*ab) found was 5 for both paintings and natural scenes, a similarity that held for the other spaces tested. In addition, the distribution of errors across the color space shows that the errors are small in the achromatic region and increase with saturation. Overall, the results indicate that the chromatic errors estimated are close to the acceptance error and therefore RGB digital cameras are able to produce quite realistic colors of complex scenarios.

“…[ 2 ]. As such, the last few years saw a tremendous rise in hyperspectral reflectance imaging spectroscopy in the cultural heritage domain [ 3 , 4 ]. This success may be attributed to the merits of the technique: it is non-invasive, portable, and allows for the wide field imaging of an artwork in under a few minutes.…”

Section: Introductionmentioning

confidence: 99%

Artificial Intelligence for Pigment Classification Task in the Short-Wave Infrared Range

Pouyet

Miteva

Rohani

et al. 2021

Hyperspectral reflectance imaging in the short-wave infrared range (SWIR, “extended NIR”, ca. 1000 to 2500 nm) has proven to provide enhanced characterization of paint materials. However, the interpretation of the results remains challenging due to the intrinsic complexity of the SWIR spectra, presenting both broad and narrow absorption features with possible overlaps. To cope with the high dimensionality and spectral complexity of such datasets acquired in the SWIR domain, one data treatment approach is tested, inspired by innovative development in the cultural heritage field: the use of a pigment spectral database (extracted from model and historical samples) combined with a deep neural network (DNN). This approach allows for multi-label pigment classification within each pixel of the data cube. Conventional Spectral Angle Mapping and DNN results obtained on both pigment reference samples and a Buddhist painting (thangka) are discussed.