Flexible multimodal camera using a light field architecture

Abstract

We present a modified conventional camera that is able to collect multimodal images in a single exposure. Utilizing a light field architecture in corifunction with multiple filters placed in the pupil plane of a main lens, we are able to digitally reconstruct synthetic images containing specific spectral, polarimetric, and other optically filtered data. The ease with which these filters can be exchanged and reconfigured provides a high degree of flexibility in the type of information that can be collected with each image. This paper explores the various tradeoffs involved in implementing a pinhole array in parallel with a pupil-plane filter array to measure multi-dimensional optical data from a scene. It also examines the design space of a pupil-plane filter array layout. Images are shown from diff erent multimodal filter layouts, and techniques to maximize resolution and minimize error in the synthetic images are proposed.

A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging

“…Optical flow [Horn and Schunck 1981] can be applied, but in practice, it is of limited utility since it is based on the brightness constancy assumption which is necessarily violated when different optical pre-filters are being used. Similar considerations hold for light-field based plenoptic imaging solutions [Horstmeyer et al 2009;Georgiev et al 2011].…”

“…2 (b), our system can be used for light-field imaging. For this case, we can omit the filter plane and the 1:1 imaging system of the filter-based design and image the output of the pickup system directly onto the sensor (although one could also combine both systems to obtain optically-filtered lightfield images as in [Horstmeyer et al 2009] effect is based on a subsampling of the main lens aperture . In the following, we explain our setup and discuss the resulting imaging geometry.…”

Spaceborne underwater imaging

“…Polarization and soundings are auxiliary sources of information that greatly simplify the needed parameter estimation. Thus, passive light-field polarization imaging [13] appears give a handle on multilayer [6,7] scattering and refraction problems. Conducting experiments to test our theory will require systems under current development such as those in Refs.…”

An overview of computational photography

“…Most typical computational optics try to improve the imaging results by controlling the aperture and capturing optically coded images, i.e., coded aperture imaging. In previous work, various finely designed modulators have been inserted into the aperture or added in front of the camera for the capture of optically coded images, including masks [3][4][5][6][7], pinhole patterns [8], lenslets [9][10][11], filters or occluders [12][13][14][15], and mirrors [16][17][18][19][20]; these elements are used in specific applications, such as capture of the light field [21], long distance camera interaction [22], defocus deblurring [23], extending the depth of field or refocusing [4][5][6][7][23][24][25], extending the field of view [26,27], confocal imaging [20], enhancing the dynamic range [13,18,28,29], extracting the depth cues [15], and capturing multi-modal visual information [30].…”