Time‐Lapse Photometric Stereo and Applications

Abstract

This paper presents a technique to recover geometry from time-lapse sequences of outdoor scenes. We build upon photometric stereo techniques to recover approximate shadowing, shading and normal components allowing us to alter the material and normals of the scene. Previous work in analyzing such images has faced two fundamental difficulties: 1. the illumination in outdoor images consists of time-varying sunlight and skylight, and 2. the motion of the sun is restricted to a near-planar arc through the sky, making surface normal recovery unstable. We develop methods to estimate the reflection component due to skylight illumination. We also show that sunlight directions are usually non-planar, thus making surface normal recovery possible. This allows us to estimate approximate surface normals for outdoor scenes using a single day of data. We demonstrate the use of these surface normals for a number of image editing applications including reflectance, lighting, and normal editing.

Potential and Limitations of Photometric Reconstruction Through a Flock of Dove Cubesats

“…The acquisition information is illustrated in figure 3. In this plot it becomes clear that the sun angles do not describe a plane, but a cone (Shen et al, 2014) as it is close to solstice. Nevertheless, this configuration is not optimal as the low sun elevation make atmospheric transfer modeling unreliable.…”

Single Day Outdoor Photometric Stereo

“…Another important issue is that, so far, most of the literature on PS has adopted a simple directional illumination model, for which optimal lighting configurations can be theoretically derived [13], [14], [15]. Until recently, no attempt had been made to model natural lighting more realistically in an outdoor setup, where lighting cannot be controlled and atmospheric effects are difficult to predict.…”

“…These measures are closely related to the rank of matrix L, which must be three for a solution to exist; that is, L T L must be nonsingular. In practice, this matrix is always full-rank, although it is often poorly conditioned [13]. In the following, we also consider the gain factor λ k in (9) as a measure of uncertainty independent of albedo and sensor noise.…”

“…Under clear skies, the MLVsl t of the model above will point nearly towards the sun, from which arrives most of the incoming light. Thus, near an equinox (worldwide), the associated MLVs are nearly coplanar [13], resulting in poor performance, fig. 5 (a).…”

One-day outdoor photometric stereo via skylight estimation

“…Shen et al [33] present an interesting analysis on the stability of the photometric stereo problem using one-day images. As the earth revolves around the sun, the angle of declination varies from 0 to 23 degrees which leads to non-planar sunlight directions in certain days of the year.…”