Discrete Modal Decomposition: a new approach for the reflectance modeling and rendering of real surfaces

Abstract

International audienceReflectance Transformation Imaging is a recent technique allowing for the measurement and the modeling of one of the most influential parameters on the appearance of a surface, namely the angular reflectance, thanks to the change in the direction of the lighting during acquisition. From these photometric stereo images (discrete data), the angular reflectance is modeled to allow both interactive and continuous relighting of the inspected surface. Two families of functions, based on polynomials and on hemispherical harmonics, are cited and used in the literature at this aim, respectively, associated to the PTM and HSH techniques. In this paper, we propose a novel method called Discrete Modal Decomposition (DMD) based on a particular and appropriate Eigen basis derived from a structural dynamic problem. The performance of the proposed method is compared with the PTM and HSH results on three real surfaces showing different reflection behaviors. Comparisons are made in terms of both visual rendering and of statistical error (local and global). The obtained results show that the DMD is more efficient in that it allows for a more accurate modeling of the angular reflectance when light-matter interaction is complex such as the presence of shadows, specularities and inter-reflections

State‐of‐the‐art in Multi‐Light Image Collections for Surface Visualization and Analysis

“…According to those metrics, on most images local interpolation (e.g., RBF) provides better results than HSH and PTM [PDJ∗18]. Looking at several works [ZD14, PLGM∗17a], it seems that, using a similar number of coefficients, PTM/HSH and DMD provide similar results. The results reported by Xu et al [XSHR18] show that the CNN based relighting presented may provide higher PSNR than PTM and BRDF based methods [HS17].…”

“…The DMD method is based on a projection on the modal bases, which are composed of elementary forms that take their origin from a structural dynamic problem. Pitard et al [PLGF∗15, PLGM∗17a] adopt this technique for modeling angular components of the local reflectance by devising new modal shapes called Reflectance Modal Basis (RMB) from the set of images based on the Discrete Modal decomposition. They model the surface reflectance function, f , as the sum of a linear combination of the modal vectors and the residue of the decomposition R n , which can be expressed as , where the modal coefficients λ k can be obtained as the projection of the vector of measured luminance L onto each modes ( Q k , k=1,…n, where n is the number of modes) of the non‐orthonormal basis.…”

“…They model the surface reflectance function, f , as the sum of a linear combination of the modal vectors and the residue of the decomposition R n , which can be expressed as , where the modal coefficients λ k can be obtained as the projection of the vector of measured luminance L onto each modes ( Q k , k=1,…n, where n is the number of modes) of the non‐orthonormal basis. A comparative analysis with other fitting techniques has shown that the DMD is well suited for approximating the complex physical behavior of light reflections, especially for shiny reflective surfaces [PLGF∗15, PLGM∗17a].…”

“…5.3) preserves the visual richness of paintings, and allows for the visualization of how the specular behaviour migrates over the painting's surface. Pitard et al [PLGM∗17a] show that a better interactive relighting model could not only improve the quality of the specular component on a jewel visualization but also allows for a better understanding of texture relief on a wall painting.…”

Surface Reflectance: An Optical Method for Multiscale Curvature Characterization of Wear on Ceramic–Metal Composites

“…RTI is probably best known in cultural heritage communities, where it is used to study surface topographies of art and cultural heritage objects [28,29]. RTI is increasingly used in industry for inspecting surface defects on smoother surfaces [30][31][32], and, more generally, for providing appearances of industrial surfaces, for understanding how they are perceived visually [33].…”