Reasoning about Photo Collections using Models of Outdoor Illumination

Abstract

Natural illumination from the sun and sky plays a significant role in the appearance of outdoor scenes. We propose the use of sophisticated outdoor illumination models, developed in the computer graphics community, for estimating appearance and timestamps from a large set of uncalibrated images of an outdoor scene. We first present an analysis of the relationship between these illumination models and the geolocation, time, surface orientation, and local visibility at a scene point. We then use this relationship to devise a data-driven method for estimating per-point albedo and local visibility information from a set of Internet photos taken under varying, unknown illuminations. Our approach significantly extends prior work on appearance estimation to work with sun-sky models, and enables new applications, such as computing timestamps for individual photos using shading information. Modeling illumination in outdoor scenesThe illumination arriving at a point in an outdoor scene depends on several key factors, including:• geographic location • time and date • surface orientation • local visibility Our model describes the irradiance incident at an outdoor scene point on a clear day as a function L(φ , λ ,t, α, n) where φ , λ are latitude and longitude, t is the time and date, n is the normal, and α is the local visibility angle. This angle α is a parameterization of local visibility based on a model of ambient occlusion proposed by Hauagge et al. [1], which models local geometry around a point as a cylindrical hole with angle α from the normal to the opening. Figure 1 shows examples of L, in the form of spheres rendered under predicted outdoor illumination at various times and α angles, at a given location on Earth. MethodA georegistered 3D point cloud built using SfM and MVS provides geographic location (φ , λ ), surface normals ( n), and a set of observed pixel values for each point (I x ). We first estimate the albedo of each point, then use the albedo to estimate lighting and capture time for each photo. Estimating Albedo. We adopt a simple Lambertian image formation model I x = ρL x where I x is the observed color of a point x in a given image I, ρ x is the (assumed constant) albedo at that point, and L x is the irradiance as defined above. Given many observations of a point I x , we derive the albedo ρ x by dividing the average observed colorOur key insight is that we can use a sun/sky model to predict illumination for a given condition, or indeed the average illumination for a given scene. For a given location, time, and visibility angle, we compute a physically-based environment map (we use the model of Hosek and Wilkie [2]) and, for each normal, integrate over the visible portion of the environment map to produce a database of spheres giving values for L at each normal direction, as illustrated in Figure 1(a-b). We then estimate expected illuminationL( n, α) as a function of normal and visibility angle by taking the average over a set of times sampled throughout the year.For each point x, we have a surface ...

Shadow Detection and Sun Direction in Photo Collections

“…Given a rough geolocation, correct orientation, and an accurate date, our constrained sun path approach is able to obtain a median error of 10.9 • , with very few errors over 25 • . This is competitive with the results of the more complicated approach of [10] on the same dataset.…”

“…Prior work has also sought to estimate illumination properties from either shadows cast on a planar surface [22,20,13,21], or from shading information [19]. In a closely related method, Hauagge et al [10] use an intrinsic image technique that considers ambient occlusion [9] to determine shading and illumination outdoors. They then compare the illumination estimate against renderings with a physically-based sun/sky model to determine sun direction.…”

Photometric Ambient Occlusion for Intrinsic Image Decomposition

“…For instance, the color and relative intensity of the direct (sun) and indirect (sky) terms vary over time. For a more in depth exploration of the effects of outdoor illumination and an extension to the algorithm presented in this work for natural illumination we refer the reader to Hauagge et al [36]. Color in .…”

Intrinsic Decomposition of Image Sequences from Local Temporal Variations

“…Bousseau et al [7] and Bell et al [5] allow individual or crowdsourced users to provide reflectance or shading annotations. Barron et al [1], Laffont et al [19,20], and Hauagge et al [13] leverage scene geometry, which is either captured with a depth sensor or reconstructed from multiple viewpoints. Bonneel et al [6] decompose video sequences and let users interactively refine the results, while Ye et al [35] propagate an existing decomposition to every frame of a video.…”