Augmenting virtual objects into a real scene requires estimating the scene illumination so that the augmented objects can become visually coherent with real objects. We propose an online technique that learns the illumination from image sequences captured by a hand-held device. We approximate the illumination with multiple linear models, and the coefficients and bandwidth parameters of the models are updated progressively in a data-driven way. Our online learning enables us to seamlessly integrate virtual objects into a real scene by rendering the objects with the estimated lights. We demonstrate that our framework can provide a high-quality global illumination result in augmented reality at interactive rates.

Photon mapping is a light transport algorithm that simulates various rendering effects (e.g., caustics) robustly, and its progressive variants, progressive photon mapping (PPM) methods, can produce a biased but consistent rendering output. PPM estimates radiance using a kernel density estimation whose parameters (bandwidths) are adjusted progressively, and this refinement enables to reduce its estimation bias. Nonetheless, many iterations (and thus a large number of photons) are often required until PPM produces nearly converged estimates. This paper proposes a post‐reconstruction that improves the performance of PPM by reducing residual errors in PPM estimates. Our key idea is to take multiple PPM estimates with multi‐level correlation structures, and fuse the input images using a weight function trained by supervised learning with maintaining the consistency of PPM. We demonstrate that our technique boosts an existing PPM technique for various rendering scenes.