Image-based Rendering with Controllable Illumination

Wong, Tien‐Tsin; Heng, Pheng‐Ann; Or, Siu-Hang; Ng, Wai-Yin

doi:10.1007/978-3-7091-6858-5_2

Cited by 82 publications

(67 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spherical harmonics (SH), the angular portion of a set of solutions to the Laplace's equations defined on a sphere, appear to be a good candidate for this purpose. Due to their appealing mathematical properties, they have been extensively applied in a great variety of topics in computer graphics, such as the modelling of BRDFs [24], early work on image-based rendering and re-lighting [25,26], BRDF shading [27], irradiance environment maps [28], precomputed radiance transfer [29,30], distant lighting [31,32] and lighting-invariant object recognition [33]. However, in the context of PTM, we note that the incoming and outgoing lights are defined only on the upper hemisphere.…”

Section: Related Workmentioning

confidence: 99%

Efficient robust image interpolation and surface properties using polynomial texture mapping

Zhang

Drew

2014

J Image Video Proc

View full text Add to dashboard Cite

Polynomial texture mapping (PTM) uses simple polynomial regression to interpolate and re-light image sets taken from a fixed camera but under different illumination directions. PTM is an extension of the classical photometric stereo (PST), replacing the simple Lambertian model employed by the latter with a polynomial one. The advantage and hence wide use of PTM is that it provides some effectiveness in interpolating appearance including more complex phenomena such as interreflections, specularities and shadowing. In addition, PTM provides estimates of surface properties, i.e., chromaticity, albedo and surface normals. The most accurate model to date utilizes multivariate Least Median of Squares (LMS) robust regression to generate a basic matte model, followed by radial basis function (RBF) interpolation to give accurate interpolants of appearance. However, robust multivariate modelling is slow. Here we show that the robust regression can find acceptably accurate inlier sets using a much less burdensome 1D LMS robust regression (or 'mode-finder'). We also show that one can produce good quality appearance interpolants, plus accurate surface properties using PTM before the additional RBF stage, provided one increases the dimensionality beyond 6D and still uses robust regression. Moreover, we model luminance and chromaticity separately, with dimensions 16 and 9 respectively. It is this separation of colour channels that allows us to maintain a relatively low dimensionality for the modelling. Another observation we show here is that in contrast to current thinking, using the original idea of polynomial terms in the lighting direction outperforms the use of hemispherical harmonics (HSH) for matte appearance modelling. For the RBF stage, we use Tikhonov regularization, which makes a substantial difference in performance. The radial functions used here are Gaussians; however, to date the Gaussian dispersion width and the value of the Tikhonov parameter have been fixed. Here we show that one can extend a theorem from graphics that generates a very fast error measure for an otherwise difficult leave-one-out error analysis. Using our extension of the theorem, we can optimize on both the Gaussian width and the Tikhonov parameter.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient robust image interpolation and surface properties using polynomial texture mapping

Zhang

Drew

2014

J Image Video Proc

View full text Add to dashboard Cite

show abstract

“…Note that the reflectance function is specified in an image-based coordinate frame, i.e., it includes non-local and geometry-induced illumination effects like the foreshortening term, interreflections and self-shadowing. As consequence, r x 0 (ω) can be considered as a 2D slice of the so-called apparent BRDF (Wong et al (1997)). …”

Section: Acquisition Of Reflectance Featuresmentioning

confidence: 99%

Reflectance Modeling in Machine Vision: Applications in Image Analysis and Synthesis

Gruna¹,

Irgenfried²

2012

Machine Vision - Applications and Systems

View full text Add to dashboard Cite

“…Using our very simple set-up and photometric stereo method, we address the inverse problem of real-time rendering by using compressed data. Image-based relighting [43], [30] is a method to achieve real-time illumination computation of arbitrarily complex scenes. It shifts the data acquisition (for a real scene) or the time-consuming illumination computation (for synthetic scenes) to a preprocessing stage and stores the results in a compact form.…”

Section: The Inverse Problem: Application In Real-time Relightingmentioning

confidence: 99%

“…During the runtime, illumination effects are achieved by real-time decompression and composition. However, if per-pixel depth information is not available, photorealistic relighting with interesting lighting effects such as illumination due to spotlight, point light source, or slide projection cannot be correctly achieved [43]. On the other hand, the necessary data for relighting basically consists of a dense image set captured under a moving distant light source, which is exactly what our dense photometric stereo needs for normal reconstruction.…”

Section: The Inverse Problem: Application In Real-time Relightingmentioning

confidence: 99%

See 1 more Smart Citation

Dense Photometric Stereo: A Markov Random Field Approach

Tang

et al. 2006

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-We address the problem of robust normal reconstruction by dense photometric stereo, in the presence of complex geometry, shadows, highlight, transparencies, variable attenuation in light intensities, and inaccurate estimation in light directions. The input is a dense set of noisy photometric images, conveniently captured by using a very simple set-up consisting of a digital video camera, a reflective mirror sphere, and a handheld spotlight. We formulate the dense photometric stereo problem as a Markov network and investigate two important inference algorithms for Markov Random Fields (MRFs)-graph cuts and belief propagation-to optimize for the most likely setting for each node in the network. In the graph cut algorithm, the MRF formulation is translated into one of energy minimization. A discontinuity-preserving metric is introduced as the compatibility function, which allows -expansion to efficiently perform the maximum a posteriori (MAP) estimation. Using the identical dense input and the same MRF formulation, our tensor belief propagation algorithm recovers faithful normal directions, preserves underlying discontinuities, improves the normal estimation from one of discrete to continuous, and drastically reduces the storage requirement and running time. Both algorithms produce comparable and very faithful normals for complex scenes. Although the discontinuity-preserving metric in graph cuts permits efficient inference of optimal discrete labels with a theoretical guarantee, our estimation algorithm using tensor belief propagation converges to comparable results, but runs faster because very compact messages are passed and combined. We present very encouraging results on normal reconstruction. A simple algorithm is proposed to reconstruct a surface from a normal map recovered by our method. With the reconstructed surface, an inverse process, known as relighting in computer graphics, is proposed to synthesize novel images of the given scene under user-specified light source and direction. The synthesis is made to run in real time by exploiting the state-of-the-art graphics processing unit (GPU). Our method offers many unique advantages over previous relighting methods and can handle a wide range of novel light sources and directions.

show abstract

Image-based Rendering with Controllable Illumination

Cited by 82 publications

References 8 publications

Efficient robust image interpolation and surface properties using polynomial texture mapping

Efficient robust image interpolation and surface properties using polynomial texture mapping

Reflectance Modeling in Machine Vision: Applications in Image Analysis and Synthesis

Dense Photometric Stereo: A Markov Random Field Approach

Contact Info

Product

Resources

About