Summary By inserting a microlens array at the intermediate image plane of an optical microscope, one can record four‐dimensional light fields of biological specimens in a single snapshot. Unlike a conventional photograph, light fields permit manipulation of viewpoint and focus after the snapshot has been taken, subject to the resolution of the camera and the diffraction limit of the optical system. By inserting a second microlens array and video projector into the microscope's illumination path, one can control the incident light field falling on the specimen in a similar way. In this paper, we describe a prototype system we have built that implements these ideas, and we demonstrate two applications for it: simulating exotic microscope illumination modalities and correcting for optical aberrations digitally.

Figure 1:The techniques in this paper employ two computer-assisted optical effects: synthetic aperture photography and synthetic aperture illumination. On the left, we aim a camera at an array of planar mirrors, yielding 22 different views of a statuette partially obscured by a plant. By rectifying, shifting, and adding these views together, we simulate a camera with a wide aperture and a shallow depth of field. Using appropriate shifts, we can position the focal plane of this synthetic camera astride the statuette, blurring out the plant. On the right we replace the camera with a video projector. By shifting, keystoning, and projecting multiple copies of a binary pattern, we produce a real image with a similarly shallow depth of field. Using appropriate shifts, we can position this image astride the statuette. On this plane the image is well focused; elsewhere, it is blurry. AbstractConfocal microscopy is a family of imaging techniques that employ focused patterned illumination and synchronized imaging to create cross-sectional views of 3D biological specimens. In this paper, we adapt confocal imaging to large-scale scenes by replacing the optical apertures used in microscopy with arrays of real or virtual video projectors and cameras. Our prototype implementation uses a video projector, a camera, and an array of mirrors. Using this implementation, we explore confocal imaging of partially occluded environments, such as foliage, and weakly scattering environments, such as murky water. We demonstrate the ability to selectively image any plane in a partially occluded environment, and to see further through murky water than is otherwise possible. By thresholding the confocal images, we extract mattes that can be used to selectively illuminate any plane in the scene.

No abstract

Figure 1:The techniques in this paper employ two computer-assisted optical effects: synthetic aperture photography and synthetic aperture illumination. On the left, we aim a camera at an array of planar mirrors, yielding 22 different views of a statuette partially obscured by a plant. By rectifying, shifting, and adding these views together, we simulate a camera with a wide aperture and a shallow depth of field. Using appropriate shifts, we can position the focal plane of this synthetic camera astride the statuette, blurring out the plant. On the right we replace the camera with a video projector. By shifting, keystoning, and projecting multiple copies of a binary pattern, we produce a real image with a similarly shallow depth of field. Using appropriate shifts, we can position this image astride the statuette. On this plane the image is well focused; elsewhere, it is blurry. AbstractConfocal microscopy is a family of imaging techniques that employ focused patterned illumination and synchronized imaging to create cross-sectional views of 3D biological specimens. In this paper, we adapt confocal imaging to large-scale scenes by replacing the optical apertures used in microscopy with arrays of real or virtual video projectors and cameras. Our prototype implementation uses a video projector, a camera, and an array of mirrors. Using this implementation, we explore confocal imaging of partially occluded environments, such as foliage, and weakly scattering environments, such as murky water. We demonstrate the ability to selectively image any plane in a partially occluded environment, and to see further through murky water than is otherwise possible. By thresholding the confocal images, we extract mattes that can be used to selectively illuminate any plane in the scene.