Refractive shape from light field distortion

Abstract

Acquiring transparent, refractive objects is challenging as these kinds of objects can only be observed by analyzing the distortion of reference background patterns. We present a new, single image approach to reconstructing thin transparent surfaces, such as thin solids or surfaces of fluids. Our method is based on observing the distortion of light field background illumination. Light field probes have the potential to encode up to four dimensions in varying colors and intensities: spatial and angular variation on the probe surface; commonly employed reference patterns are only two-dimensional by coding either position or angle on the probe. We show that the additional information can be used to reconstruct refractive surface normals and a sparse set of control points from a single photograph.

Reconstructing Gas Flows Using Light-Path Approximation

“…For example, they propose using Bokode [19], a special pinhole projector, to acquire ray-ray correspondences for directly recovering fluid surface normals. Closest to our approach, Wetzstein et al [29] replace the conventional checkerboard pattern with a LF-probe with 4D spatial and angular coding to obtain more accurate ray-ray correspondences. We study non-linear light paths using ray-ray correspondences.…”

“…The core of our approach is to acquire ray-ray correspondences using the LF-probe [28,29]. A LF-probe, as shown in Fig.…”

“…Recent studies have shown that ray-ray correspondences greatly benefit specular surface reconstruction. For example, assuming single refraction through uniform media, Wetzstein et al [29] and Ye et al [30] use ray-ray correspondences to infer surface heights and normal fields of 2D dynamic wavefronts. We demonstrate how to use ray-ray correspondences for inferring light paths [16] through the RIF within the gas volume.…”

“…Our solution exploits the light field probe (LF-Probe) [28,29] which serves as a view-dependent reference pattern. A LF-probe, in essence, is an "inverted" light field camera [22] where each pixel on the pattern maps to a unique ray.…”

Refraction Wiggles for Measuring Fluid Depth and Velocity from Video

“…Tian et al [27] showed that atmospheric turbulence provides a depth cue as larger deflections due to refraction typically correspond to greater depths. Wetzstein et al [29] proposed to recover the shape of a refractive solid object from light field distortions. Alterman et al [2,3] showed that it is possible to locate a moving object through refractive atmospheric turbulence or a water interface.…”