Dual photography

Abstract

We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, inter-reflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.

Non-Parametric Acquisition of Near-Dirac Pixel Correspondences

“…Most employ strategies similar to those used in environment matting. Sen et al (2005) propose a hierarchical decomposition into non-interfering regions. The adaptive approach requires many images to resolve PSFs overlapping multiple regions.…”

“…Previous methods for handling boundary overlaps in tile-based schemes have involved scanning additional passes with translationally offset tile grids (Sen et al, 2005) and considering only one of these passes: that which finds the PSF fully enclosed by the frequency layout, our method requires only a single pass, as long as the PSF is smaller than a single tile. We locate the maximum value in the magnitude spectrum and circularly shift this to the centre of the tile, recording the shift vector so that we can subtract it and still obtain an absolute position in global coordinates.…”

A Dual Theory of Inverse and Forward Light Transport

“…3, the difference is only in the first bounce or direct lighting, as interactions within the scene are still governed by the rendering equation. Extensions to incident (and reflected) light fields [Masselus et al 2003;Sen et al 2005;Garg et al 2006] are encompassed by the theory, but not yet considered in our practical applications.…”

“…An accurate, one-time, radiometric calibration of the projector and camera response curves is performed to ensure linearity of the corresponding signals. While prior work has obtained transport matrices at resolutions comparable to ours [Sen et al 2005], it has mainly been for applications akin to relighting. In contrast, the inverse problems that form our application domain require greater fidelity in the elements of the transport matrix.…”

“…The projector's black offset is computed at the highest exposure to average out high frequency fluctuations. For the higher resolution scenes, a hierarchical subdivision scheme is used to simultaneously acquire portions of the transport matrix which are not in mutual conflict, similar to [Sen et al 2005].…”