An integrated imaging sensor for aberration-corrected 3D photography

Abstract

Planar digital image sensors facilitate broad applications in a wide range of areas1–5, and the number of pixels has scaled up rapidly in recent years2,6. However, the practical performance of imaging systems is fundamentally limited by spatially nonuniform optical aberrations originating from imperfect lenses or environmental disturbances7,8. Here we propose an integrated scanning light-field imaging sensor, termed a meta-imaging sensor, to achieve high-speed aberration-corrected three-dimensional photography for universal applications without additional hardware modifications. Instead of directly detecting a two-dimensional intensity projection, the meta-imaging sensor captures extra-fine four-dimensional light-field distributions through a vibrating coded microlens array, enabling flexible and precise synthesis of complex-field-modulated images in post-processing. Using the sensor, we achieve high-performance photography up to a gigapixel with a single spherical lens without a data prior, leading to orders-of-magnitude reductions in system capacity and costs for optical imaging. Even in the presence of dynamic atmosphere turbulence, the meta-imaging sensor enables multisite aberration correction across 1,000 arcseconds on an 80-centimetre ground-based telescope without reducing the acquisition speed, paving the way for high-resolution synoptic sky surveys. Moreover, high-density accurate depth maps can be retrieved simultaneously, facilitating diverse applications from autonomous driving to industrial inspections.

Multi-focus light-field microscopy for high-speed large-volume imaging

“…To meet the pressing demand, various efforts have been invested during the past decades to develop fast and high-quality volumetric imaging methods [13][14][15][16], among which light-field microscopy (LFM) [17] is an attractive candidate due to its high parallelization and low phototoxicity. By inserting a microlens array (MLA) into the detection path, LFM encodes the high-dimensional information of a large volume in a snapshot, providing a powerful capability of high-speed 3D imaging [18][19][20]. However, similar to common microscopy techniques, LFM still suffers from the tradeoff between spatial-temporal resolution and the volume coverage, which impedes observing subtle details in large-scale volumes.…”

Large-scale Global Low-rank Optimization for Computational Compressed Imaging

“…By jointly optimizing computation and optics, computational imaging [1] improves the efficiency and information capacity of optical systems [2]. With computational imaging, one can achieve detection under extremely complex lighting conditions [3,4], capture invisible high-dimensional information [4][5][6], look through obstacles [7,8], etc.…”

Virtual-scanning light-field microscopy for robust snapshot high-resolution volumetric imaging

“…Although numerous reconstruction algorithms have been developed to enable the practical and versatile application of LFM in biology [21][22][23][24][25][26][27] , LFM is still hindered by low spatial resolution and reconstruction artifacts, especially in complicated intravital environments. By introducing periodic beam drifting to increase the spatial sampling density, scanning LFM (sLFM) increases the resolution up to the diffraction limit and facilitates multi-site aberration correction in post-processing 28 , but the physical scanning process reduces the 3D imaging speed and may introduce motion artifacts for highly dynamic samples 20 .…”