Learning Wavefront Coding for Extended Depth of Field Imaging

Abstract

The depth of field constitutes an important quality factor of imaging systems that highly affects the content of the acquired spatial information in the captured images. Extended depth of field (EDoF) imaging is a challenging problem due to its highly ill-posed nature, hence it has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which helps in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.

Learning an optimal PSF-pair for ultra-dense 3D localization microscopy

“…Notably, recent end-to-end designs of EDOF PSFs have achieved quite compelling results [52][53][54]. In particular, the phase mask presented in [54] resembles the result of our approach.…”

“…Notably, recent end-to-end designs of EDOF PSFs have achieved quite compelling results [52][53][54]. In particular, the phase mask presented in [54] resembles the result of our approach. However, these data-driven approaches are ultimately dataset-dependant, and take hours of training to design for a new range, whereas our approach is independent of the dataset and converges in less than 2 minutes on GPU.…”

“…Powered by differentiable imaging models and back-propagation, end-toend learning jointly optimizes the sensing system alongside the data-processing algorithm, thus enabling both components to work harmoniously. This approach has quickly expanded within the computational-imaging community for numerous applications in computer vision and computational photography, for example, color sensing and demosaicing [49,50], illuminationdesign through scattering media [51], extended-depth-of-field imaging [52][53][54], monocular depth estimation [52,53,55,56], high-dynamic-range imaging [57,58], and hyper-spectral imaging [59,60]. In computational microscopy, end-to-end learning has been utilized by our group and others to enhance various computational modalities such as sample classification [31,32], single-molecule color sensing and 3D localization [21,41], quantitative phase imaging [61] and multi-photon microscopy [62].…”