Enhanced angle sensitive pixels for light field imaging

Sivaramakrishnan, Sriram; Wang, Albert; Gill, Patrick R.; Molnar, Alyosha

doi:10.1109/iedm.2011.6131516

Cited by 10 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Whereas initial ASP sensor designs use two layered, attenuating diffraction gratings and conventional photodiodes underneath [31,32,11], more recent versions enhance the quantum efficiency of the design by using a single phase grating and an interleaved pair of photodiodes [27]. For the proposed switchable light field camera, we illustrate the latter design with the layout of a single pixel in Figure 2.…”

Section: The Integral In Equationmentioning

confidence: 99%

“…The Talbot effect created by periodic gratings induces a sinusoidal angular response from ASPs [27]. For a onedimensional ASP, this can be described as…”

Section: Light Field Acquisition With Aspsmentioning

confidence: 99%

“…Creating a sensor of tiled ASPs with pre-selected responses enables range imaging, focal stacks [32], and lensless imaging [11]. Optically optimized devices, created with phase gratings and multiple interdigitated diodes can achieve quantum efficiency comparable to standard CMOS imagers [27].…”

Section: Angle Sensitive Pixelsmentioning

confidence: 99%

“…The physical principle behind ASPs is the Talbot effect: light incident on a pixel strikes two periodic diffraction gratings that are manufactured using commodity CMOS processes at a slight offset in front of the photodiodes. Whereas several ASP chip designs have been proposed in previous work [31,27], we combine ASP hardware with modern techniques for compressive light field reconstruction and other processing modes into what we believe to be the most flexible light field camera architecture to date.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding

Hirsch

Sivaramakrishnan

Jayasuriya

et al. 2014

2014 IEEE International Conference on Computational Photography (ICCP)

Self Cite

View full text Add to dashboard Cite

We propose a flexible light field camera architecture that is at the convergence of optics, sensor electronics, and applied mathematics. Through the co-design of a sensor that comprises tailored, Angle Sensitive Pixels and advanced reconstruction algorithms, we show that-contrary to light field cameras today-our system can use the same measurements captured in a single sensor image to recover either a high-resolution 2D image, a low-resolution 4D light field using fast, linear processing, or a high-resolution light field using sparsity-constrained optimization.

show abstract

Section: The Integral In Equationmentioning

confidence: 99%

“…The Talbot effect created by periodic gratings induces a sinusoidal angular response from ASPs [27]. For a onedimensional ASP, this can be described as…”

Section: Light Field Acquisition With Aspsmentioning

confidence: 99%

Section: Angle Sensitive Pixelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding

Hirsch

Sivaramakrishnan

Jayasuriya

et al. 2014

2014 IEEE International Conference on Computational Photography (ICCP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…While these methods are popular techniques for hyperspectral acquisition, they are often limited to static scenes and result in objectionable motion-related artifacts in dynamic scenes. Generalized Assorted Pixels (GAP) is slowly gaining popularity as a method for acquiring spectral [3], [4], [5], polarization [3], [4], and angular information [6] on a single image sensor. Unfortunately, several challenges to its widespread adoption remain: Fabrication-GAP requires nano-scale manufacturing techniques that can produce filters in a manner that is compatible with semiconductor fabrication processes, Cost-currently available GAP sensors such as those offered by PixelTeq and IMEC are expensive, and Resolution-the use of GAP results in loss of spatial resolution and often produces low-resolution hyperspectral images (e.g., 256 × 256 resolution in IMEC).…”

Section: Introductionmentioning

confidence: 99%

Generalized Assorted Camera Arrays: Robust Cross-Channel Registration and Applications

Holloway

Mitra

Koppal

et al. 2015

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Abstract-One popular technique for multi-modal imaging is Generalized Assorted Pixels (GAP), where an assorted pixel array on the image sensor allows for multi-modal capture. Unfortunately GAP is limited in its applicability because of the need for multi-modal filters that are amenable with semiconductor fabrication processes and results in a fixed multi-modal imaging configuration. In this paper, we advocate for Generalized Assorted Camera (GAC) arrays for multi-modal imaging-i.e., a camera array with filters of different characteristics placed in front of each camera aperture. GAC provides us with three distinct advantages over GAP: ease of implementation, flexible application dependent imaging since filters are external and can be changed and depth information that can be used for enabling novel applications (e.g. post-capture refocusing). The primary challenge in GAC arrays is that since the different modalities are obtained from different viewpoints, there is a need for accurate and efficient cross-channel registration. Traditional approaches such as SSD, SAD, and mutual information all result in multimodal registration errors. Here, we propose a robust crosschannel matching cost function, based on aligning normalized gradients, that allows us to compute cross-channel sub-pixel correspondences for scenes exhibiting non-trivial geometry. We highlight the promise of GAC arrays with our cross-channel normalized gradient cost for several applications such as low light imaging, post-capture refocusing, skin perfusion imaging using RGB+NIR and hyperspectral imaging.

show abstract

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

Jayasuriya

Pediredla

Sivaramakrishnan

et al. 2015

2015 International Conference on 3D Vision

Self Cite

View full text Add to dashboard Cite

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.

show abstract

Enhanced angle sensitive pixels for light field imaging

Cited by 10 publications

References 8 publications

A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding

A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding

Generalized Assorted Camera Arrays: Robust Cross-Channel Registration and Applications

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

Contact Info

Product

Resources

About