Recovering spectral reflectance under commonly available lighting conditions

Abstract

Recovering the spectral reflectance of a scene is important for scene understanding. Previous approaches use either specialized filters or controlled illumination where the extra hardware prevents many practical applications. In this paper, we propose a method that accurately recovers spectral reflectance from two images taken with conventional consumer cameras under commonly available lighting conditions, such as daylight at different times over a day, camera flash and ambient light, and fluorescent and tungsten light. Our approach does not require camera spectral sensitivities or the spectra of the illumination, which makes it easy to implement for a variety of practical applications. Based on noise analysis, we also derive theoretical predictors that answer: (1) which two lighting conditions lead to the most accurate spectral recovery overall, and (2) for two given lighting conditions, which spectral reflectance is more likely to be estimated accurately. We implement the method on a variety of cameras from high-end DSLRs to cellphone cameras, and apply the recovered spectral reflectance for several applications such as fine art reproduction, fruit identification, and material classification. Both simulation and experimental results demonstrate the effectiveness of the proposed method.

Reflectance and Fluorescent Spectra Recovery Based on Fluorescent Chromaticity Invariance under Varying Illumination

“…In this section, we quantitatively evaluate the errors in relighting of real scenes. Figures 7,8,and 9 show relighting results using the spectral reflectance, fluorescence absorption and emission spectra estimated by our method. We evaluated the accuracy of relighting with our method by computing errors between the ground truth and the relighted results for 3 scenes under 5 illuminants as n (p gt n − p re n ) 2 / n (p gt n ) 2 , where p gt n is the ground truth (corresponding images in Figures 7-9 (b)) and p re n is the relighting result (Figures 7-9 (d) and (f)).…”

“…Determining α would allow for recovering the spectral reflectance s according to Equation (8). Finding E would provide us with the fluorescent chromaticity which will be used in later steps to determine the fluorescence spectral components.…”

“…A number of methods for capturing only the spectral reflectance of scenes using conventional RGB cameras have been proposed [3,12,8]. These methods are practical and effective for imaging spectral reflectance but their limitation is they cannot accurately capture scenes with fluorescent surfaces.…”

Reflectance and Fluorescence Spectral Recovery via Actively Lit RGB Images

“…Han et al [15] recovered spectral reflectance using multiple illuminations produced by the color wheel in a DLP projector. Jiang and Gu [16] recovered spectral reflectance from two images taken with conventional consumer cameras under commonly available lighting conditions. These methods are practical and effective for imaging spectral reflectance but their limitation is they cannot accurately capture scenes with fluorescent surfaces.…”

“…The parameters α and E need to be chosen to minimize the error function in Equation (16). Determining α would allow for recovering the spectral reflectance s according to Equation (12).…”