Learning to Estimate Indoor Lighting from 3D Objects

Abstract

In this work, we propose a step towards a more accurate prediction of the environment light given a single picture of a known object. To achieve this, we developed a deep learning method that is able to encode the latent space of indoor lighting using few parameters and that is trained on a database of environment maps. This latent space is then used to generate predictions of the light that are both more realistic and accurate than previous methods. To achieve this, our first contribution is a deep autoencoder which is capable of learning the feature space that compactly models lighting. Our second contribution is a convolutional neural network that predicts the light from a single image of an object with known geometry and reflectance. To train these networks, our third contribution is a novel dataset that contains 21,000 HDR indoor environment maps. The results indicate that the predictor can generate plausible lighting estimations even from diffuse objects.

Self-Calibrating Deep Photometric Stereo Networks

“…Hold-Goeffroy et al [14] learned outdoor lighting using a physically-based sky model. Weber et al [32] estimated indoor environment lighting from an image of an object with known shape. Zhou et al [37] estimated lighting, in the form of Spherical Harmonics, from a human face image by assuming a Lambertian reflectance model.…”

Fast Spatially-Varying Indoor Lighting Estimation

“…Inverting this image formation process to recover lighting (or any of these other intrinsic properties) is severely underconstrained. Typical solutions to this problem rely on inserting an object (a light probe) with known geometry and/or reflectance properties in the scene (a shiny sphere [5], or 3D objects of known geometry [9,34]). Unfortunately, having to insert a known object in the scene is limiting and thus not easily amenable to practical applications.…”

Neural Illumination: Lighting Prediction for Indoor Environments

“…Debevec et al [3] first showed that photographs of a mirrored sphere with different exposures can be used to compute the illumination at the sphere's location. Subsequent works show that beyond mirrored spheres, it is also possible to capture illumination using hybrid spheres [4], known 3D objects [24], object's with know surface material [8], or even human faces [1] as proxies for light probes.…”