Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network

Bostan, Emrah; Heckel, Reinhard; Chen, Michael; Kellman, Michael; Waller, Laura

doi:10.1364/optica.389314

Cited by 131 publications

(80 citation statements)

References 32 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Differentiable model-based approaches for image reconstruction have been introduced in several domains of imaging [21], for example in ptychography [31]. Instead of directly optimizing model parameters, an additional deep neural network (a deep image prior [41,42]) has also been introduced, for example for phase imaging [32,33] or ptychography [34]. Even without such additional regularization, the optimization converged reliably to smooth phase patterns (Fig.…”

Section: Discussionmentioning

confidence: 99%

“…Similar situations where models based on a well known underlying physical process are learned from data are also encountered in other imaging modalities [21], and more broadly many areas of engineering and physics (for example [22][23][24][25][26][27][28][29][30][31][32][33][34]). To take advantage of such prior information, methods have been developed that combine physical process models with machine learning optimization.…”

Section: Introductionmentioning

confidence: 99%

“…Automatic differentiation is used in these frameworks to compute gradients for optimization of a loss function with respect to parameters of interest. The loss function compares model output to a target output and the discrepancy is minimized by adjusting model parameters [22][23][24][25][26][27][28][29][30][31][32][33][34]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Differentiable model-based adaptive optics with transmitted and reflected light

Vishniakou¹,

Seelig²

2020

Opt. Express

View full text Add to dashboard Cite

Aberrations limit optical systems in many situations, for example when imaging in biological tissue. Machine learning offers novel ways to improve imaging under such conditions by learning inverse models of aberrations. Learning requires datasets that cover a wide range of possible aberrations, which however becomes limiting for more strongly scattering samples, and does not take advantage of prior information about the imaging process. Here, we show that combining model-based adaptive optics with the optimization techniques of machine learning frameworks can find aberration corrections with a small number of measurements. Corrections are determined in a transmission configuration through a single aberrating layer and in a reflection configuration through two different layers at the same time. Additionally, corrections are not limited by a predetermined model of aberrations (such as combinations of Zernike modes). Focusing in transmission can be achieved based only on reflected light, compatible with an epidetection imaging configuration.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Differentiable model-based adaptive optics with transmitted and reflected light

Vishniakou¹,

Seelig²

2020

Opt. Express

View full text Add to dashboard Cite

show abstract

“…Generally, neural networks (NNs) are more and more regularly used for computer vision and related machine learning tasks. In microscopy, a whole range of application areas for NNs has emerged [11], including denoising [12,13], digital staining [14][15][16][17], counting and labelling [18], tracking [19], image reconstruction [20][21][22], computational microscopy [23][24][25][26], virtual focusing [27,28], aberration estimation [29], and segmentation [30][31][32]. In the context of FPM, attempts have been made to perform the whole phase retrieval process with a neural network although using neural networks for the full FPM reconstruction pipeline is still an area of active research.…”

Section: Theory and Methodsmentioning

confidence: 99%

Object detection neural network improves Fourier ptychography reconstruction

et al. 2020

View full text Add to dashboard Cite

High resolution microscopy is heavily dependent on superb optical elements and superresolution microscopy even more so. Correcting unavoidable optical aberrations during post-processing is an elegant method to reduce the optical system's complexity. A prime method that promises superresolution, aberration correction, and quantitative phase imaging is Fourier ptychography. This microscopy technique combines many images of the sample, recorded at differing illumination angles akin to computed tomography and uses error minimisation between the recorded images with those generated by a forward model. The more precise knowledge of those illumination angles is available for the image formation forward model, the better the result. Therefore, illumination estimation from the raw data is an important step and supports correct phase recovery and aberration correction. Here, we derive how illumination estimation can be cast as an object detection problem that permits the use of a fast convolutional neural network (CNN) for this task. We find that faster-RCNN delivers highly robust results and outperforms classical approaches by far with an up to 3-fold reduction in estimation errors. Intriguingly, we find that conventionally beneficial smoothing and filtering of raw data is counterproductive in this type of application. We present a detailed analysis of the network's performance and provide all our developed software openly.

show abstract

“…Following this idea, related methods can work with limited data 29,30 and can work stably toward system aberration. [31][32][33][34] However, these works are still essentially the iterative-based algorithm, and the automatic differentiation (AD) property of the neural network is not fully utilized. It would be much more desirable to design a new neural network to further degrade the noise in the reconstruction and estimate the optical aberration with higher accuracy.…”

Section: Introductionmentioning

confidence: 99%

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

et al. 2021

View full text Add to dashboard Cite

Significance: Fourier ptychography (FP) is a computational imaging approach that achieves high-resolution reconstruction. Inspired by neural networks, many deep-learning-based methods are proposed to solve FP problems. However, the performance of FP still suffers from optical aberration, which needs to be considered. Aim:We present a neural network model for FP reconstructions that can make proper estimation toward aberration and achieve artifact-free reconstruction.Approach: Inspired by the iterative reconstruction of FP, we design a neural network model that mimics the forward imaging process of FP via TensorFlow. The sample and aberration are considered as learnable weights and optimized through back-propagation. Especially, we employ the Zernike terms instead of aberration to decrease the optimization freedom of pupil recovery and perform a high-accuracy estimation. Owing to the auto-differentiation capabilities of the neural network, we additionally utilize total variation regularization to improve the visual quality.Results: We validate the performance of the reported method via both simulation and experiment. Our method exhibits higher robustness against sophisticated optical aberrations and achieves better image quality by reducing artifacts. Conclusions:The forward neural network model can jointly recover the high-resolution sample and optical aberration in iterative FP reconstruction. We hope our method that can provide a neural-network perspective to solve iterative-based coherent or incoherent imaging problems.

show abstract

Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network

Cited by 131 publications

References 32 publications

Differentiable model-based adaptive optics with transmitted and reflected light

Differentiable model-based adaptive optics with transmitted and reflected light

Object detection neural network improves Fourier ptychography reconstruction

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

Contact Info

Product

Resources

About