We present a method for computing ambient occlusion (AO) for a stack of images of a scene from a fixed viewpoint. Ambient occlusion, a concept common in computer graphics, characterizes the local visibility at a point: it approximates how much light can reach that point from different directions without getting blocked by other geometry. While AO has received surprisingly little attention in vision, we show that it can be approximated using simple, per-pixel statistics over image stacks, based on a simplified image formation model. We use our derived AO measure to compute reflectance and illumination for objects without relying on additional smoothness priors, and demonstrate state-of-the art performance on the MIT Intrinsic Images benchmark. We also demonstrate our method on several synthetic and real scenes, including 3D printed objects with known ground truth geometry.

Natural illumination from the sun and sky plays a significant role in the appearance of outdoor scenes. We propose the use of sophisticated outdoor illumination models, developed in the computer graphics community, for estimating appearance and timestamps from a large set of uncalibrated images of an outdoor scene. We first present an analysis of the relationship between these illumination models and the geolocation, time, surface orientation, and local visibility at a scene point. We then use this relationship to devise a data-driven method for estimating per-point albedo and local visibility information from a set of Internet photos taken under varying, unknown illuminations. Our approach significantly extends prior work on appearance estimation to work with sun-sky models, and enables new applications, such as computing timestamps for individual photos using shading information. Modeling illumination in outdoor scenesThe illumination arriving at a point in an outdoor scene depends on several key factors, including:• geographic location • time and date • surface orientation • local visibility Our model describes the irradiance incident at an outdoor scene point on a clear day as a function L(φ , λ ,t, α, n) where φ , λ are latitude and longitude, t is the time and date, n is the normal, and α is the local visibility angle. This angle α is a parameterization of local visibility based on a model of ambient occlusion proposed by Hauagge et al. [1], which models local geometry around a point as a cylindrical hole with angle α from the normal to the opening. Figure 1 shows examples of L, in the form of spheres rendered under predicted outdoor illumination at various times and α angles, at a given location on Earth. MethodA georegistered 3D point cloud built using SfM and MVS provides geographic location (φ , λ ), surface normals ( n), and a set of observed pixel values for each point (I x ). We first estimate the albedo of each point, then use the albedo to estimate lighting and capture time for each photo. Estimating Albedo. We adopt a simple Lambertian image formation model I x = ρL x where I x is the observed color of a point x in a given image I, ρ x is the (assumed constant) albedo at that point, and L x is the irradiance as defined above. Given many observations of a point I x , we derive the albedo ρ x by dividing the average observed colorOur key insight is that we can use a sun/sky model to predict illumination for a given condition, or indeed the average illumination for a given scene. For a given location, time, and visibility angle, we compute a physically-based environment map (we use the model of Hosek and Wilkie [2]) and, for each normal, integrate over the visible portion of the environment map to produce a database of spheres giving values for L at each normal direction, as illustrated in Figure 1(a-b). We then estimate expected illuminationL( n, α) as a function of normal and visibility angle by taking the average over a set of times sampled throughout the year.For each point x, we have a surface ...

Many of today's most successful video segmentation methods use long-term feature trajectories as their first processing step. Such methods typically use spectral clustering to segment these trajectories, implicitly assuming that motion is translational in image space. In this paper, we explore the idea of explicitly fitting more general motion models in order to classify trajectories as foreground or background. We find that homographies are sufficient to model a wide variety of background motions found in real-world videos. Our simple approach achieves competitive performance on the DAVIS benchmark, while using techniques complementary to state-of-the-art approaches.

Abstract-We present a method for computing ambient occlusion (AO) for a stack of images of a Lambertian scene from a fixed viewpoint. Ambient occlusion, a concept common in computer graphics, characterizes the local visibility at a point: it approximates how much light can reach that point from different directions without getting blocked by other geometry. While AO has received surprisingly little attention in vision, we show that it can be approximated using simple, per-pixel statistics over image stacks, based on a simplified image formation model. We use our derived AO measure to compute reflectance and illumination for objects without relying on additional smoothness priors, and demonstrate state-of-the art performance on the MIT Intrinsic Images benchmark. We also demonstrate our method on several synthetic and real scenes, including 3D printed objects with known ground truth geometry.

Modeling the appearance of outdoor scenes from photo collections is challenging because of appearance variation, especially due to illumination. In this paper we present a simple and robust algorithm for estimating illumination properties-shadows and sun direction-from photo collections. These properties are key to a variety of scene modeling applications, including outdoor intrinsic images, realistic 3D scene rendering, and temporally varying (4D) reconstruction. Our shadow detection method uses illumination ratios to analyze lighting independent of camera effects, and determines shadow labels for each 3D point in a reconstruction. These shadow labels can then be used to detect shadow boundaries and estimate sun direction, as well as to compute dense shadow labels in pixel space. We demonstrate our method on large Internet photo collections of scenes, and show that it outperforms prior multi-image shadow detection and sun direction estimation methods.