Multi‐Light Image Collections (MLICs), i.e., stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination, provide large amounts of visual and geometric information. In this survey, we provide an up‐to‐date integrative view of MLICs as a mean to gain insight on objects through the analysis and visualization of the acquired data. After a general overview of MLICs capturing and storage, we focus on the main approaches to produce representations usable for visualization and analysis. In this context, we first discuss methods for direct exploration of the raw data. We then summarize approaches that strive to emphasize shape and material details by fusing all acquisitions in a single enhanced image. Subsequently, we focus on approaches that produce relightable images through intermediate representations. This can be done both by fitting various analytic forms of the light transform function, or by locally estimating the parameters of physically plausible models of shape and reflectance and using them for visualization and analysis. We finally review techniques that improve object understanding by using illustrative approaches to enhance relightable models, or by extracting features and derived maps. We also review how these methods are applied in several, main application domains, and what are the available tools to perform MLIC visualization and analysis. We finally point out relevant research issues, analyze research trends, and offer guidelines for practical applications.

We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.

Photo-sensitive materials tend to change with exposure to light. Often, this change is visible when it affects the reflectance of the material in the visible range of the electromagnetic spectrum. In order to understand the photo-degradation mechanisms and their impact on fugitive materials, high-end scientific analysis is required. In a two-part article, we present a multi-modal approach to model fading effects in the spectral, temporal (first part) and spatial dimensions (second part). Specifically, we collect data from the same artwork, namely “A Japanese Lantern” by Norwegian artist, Oda Krohg, with two techniques, point-based microfading spectroscopy and hyperspectral imaging. In this first part, we focus on characterizing the pigments in the painting based on their spectral and fading characteristics. To begin with, using microfading data of a region in the painting, we analyze the color deterioration of the measured points. Then, we train a tensor decomposition model to reduce the measured materials to a spectral basis of unmixed pigments and, at the same time, to recover the fading rate of these endmembers (i.e. pure, unmixed chemical signals). Afterwards, we apply linear regression to predict the fading rate in the future. We validate the quality of these predictions by spectrally comparing them with temporal observations not included in the training part. Furthermore, we statistically assess the goodness of our model in explaining new data, collected from another region of the painting. Finally, we propose a visual way to explore the artist’s palette, where potential matches between endmembers and reference spectral libraries can be evaluated based on three metrics at once.

No abstract

Fluorescent or luminescent materials absorb light at shorter wavelengths and re-emit at longer wavelengths. In terms of appearance, this translates to a increased color vividness, as well as a glowing effect. Thanks to these properties, the study of fluorescent materials is of interest for Cultural Heritage applications, where fluorophores are incorporated in many pigments and varnishes. This paper proposes a variant to Reflectance Transformation Imaging (RTI) technique, namely Fluorescence Transformation Imaging (FTI), that handles luminescent objects. Reflectance Transformation Imaging method outputs a single-camera multi-light image collection of a static scene, which can be used to model the reflectance of the scene as a polynomial of the illumination directions. Similarly, Fluorescence Transformation Imaging aims to model the fluorescent signal based on a series of images with fixed scene and viewpoint and varying incident light directions -what changes with respect to RTI is that fluorescence is recorded instead of reflected radiation. In the literature, there are works that explore the isotropic property of fluorescence in low-dimension multi-light imagery methods (such as Photometric Stereo) to model the appearance of an object with a first-order polynomial. This is based on the assumption that in the fluorescent mode the object gets closer to a Lambertian surface than in the reflective mode where non-Lambertian effects such as highlights are more likely to appear. Nonetheless, this assumption stands for single-object scenes, with uniform albedo and convex geometries. When there are multiple fluorescent objects in the scene, with concavities and non-uniform fluorescent component, then the fluorescence can become secondary light to the object and become a source of interreflections. Through quantitative and qualitative analysis, this paper explores the Reflectance and Fluorescence Transformation Imaging methods and the resulting texture maps towards appearance rendering of heterogeneous non-flat fluorescent objects.