Neural network model combined with pupil recovery for Fourier ptychographic microscopy

Sun, Minglu; Chen, Xiong; Zhu, Youqiang; Li, Dayu; Mu, Quanquan; Li, Xuan

doi:10.1364/oe.27.024161

Cited by 28 publications

(14 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to quantitatively evaluate the performance of these two methods from the reconstructed spectrum, we calculate the normalized mean square error (NMSE) between the L 1 ‐norms of the reconstructed spectra and the ground truth spectrum as shown in Table 1 [9, 10]. We can see that DFNN presents the best result in NMSE, which means the reconstructed spectrum is more similar to the ground truth.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Sun

Shao

Zhu

et al. 2021

Journal of Biophotonics

Self Cite

View full text Add to dashboard Cite

Fourier ptychographic microscopy is a promising imaging technique which can circumvent the space‐bandwidth product of the system and achieve a reconstruction result with wide field‐of‐view (FOV), high‐resolution and quantitative phase information. However, traditional iterative‐based methods typically require multiple times to get convergence, and due to the wave vector deviation in different areas, the millimeter‐level full‐FOV cannot be well reconstructed once and typically required to be separated into several portions with sufficient overlaps and reconstructed separately, which makes traditional methods suffer from long reconstruction time for a large‐FOV (of the order of minutes) and limits the application in real‐time large‐FOV monitoring of live sample in vitro. Here we propose a novel deep‐learning based method called DFNN which can be used in place of traditional iterative‐based methods to increase the quality of single large‐FOV reconstruction and reducing the processing time from 167.5 to 0.1125 second. In addition, we demonstrate that by training based on the simulation dataset with high‐entropy property (Opt. Express 28, 24 152 [2020]), DFNN could has fine generalizability and little dependence on the morphological features of samples. The superior robustness of DFNN against noise is also demonstrated in both simulation and experiment. Furthermore, our model shows more robustness against the wave vector deviation. Therefore, we could achieve better results at the edge areas of a single large‐FOV reconstruction. Our method demonstrates a promising way to perform real‐time single large‐FOV reconstructions and provides further possibilities for real‐time large‐FOV monitoring of live samples with sub‐cellular resolution.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Several methods for suppressing the negative influence of the background noise have been proposed [4–8]. By embedding the pupil recovery procedure, the impacts of the optical aberrations could be eliminated [9, 10]. In addition, many new optimization methods have been developed to improve the robustness of FPM [11–16].…”

Section: Introductionmentioning

confidence: 99%

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Sun

Shao

Zhu

et al. 2021

Journal of Biophotonics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Following this idea, related methods can work with limited data 29,30 and can work stably toward system aberration. [31][32][33][34] However, these works are still essentially the iterative-based algorithm, and the automatic differentiation (AD) property of the neural network is not fully utilized. It would be much more desirable to design a new neural network to further degrade the noise in the reconstruction and estimate the optical aberration with higher accuracy.…”

Section: Introductionmentioning

confidence: 99%

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

et al. 2021

View full text Add to dashboard Cite

Significance: Fourier ptychography (FP) is a computational imaging approach that achieves high-resolution reconstruction. Inspired by neural networks, many deep-learning-based methods are proposed to solve FP problems. However, the performance of FP still suffers from optical aberration, which needs to be considered. Aim:We present a neural network model for FP reconstructions that can make proper estimation toward aberration and achieve artifact-free reconstruction.Approach: Inspired by the iterative reconstruction of FP, we design a neural network model that mimics the forward imaging process of FP via TensorFlow. The sample and aberration are considered as learnable weights and optimized through back-propagation. Especially, we employ the Zernike terms instead of aberration to decrease the optimization freedom of pupil recovery and perform a high-accuracy estimation. Owing to the auto-differentiation capabilities of the neural network, we additionally utilize total variation regularization to improve the visual quality.Results: We validate the performance of the reported method via both simulation and experiment. Our method exhibits higher robustness against sophisticated optical aberrations and achieves better image quality by reducing artifacts. Conclusions:The forward neural network model can jointly recover the high-resolution sample and optical aberration in iterative FP reconstruction. We hope our method that can provide a neural-network perspective to solve iterative-based coherent or incoherent imaging problems.

show abstract

“…In recent years, using data-driven method or neural network in the process of imaging and reconstruction is a trend of FP-related algorithms. [29][30][31][32][33][34][35][36] These developments can be roughly divided into four types, and more discussions can be found in the recent review article. 37 The end-to-end convolutional neural network (CNN) was proposed to learn the mapping from a stack of low resolution images to the high-resolution image.…”

Section: Introductionmentioning

confidence: 99%

“…38 Inspired by Zheng's method, Sun et al proposed a forward imaging neural network algorithm with pupil recovery function (FINN-P). 35 In addition to the object, FINN-P modeled the pupil function (or CTF) as the layer's weights as well. FINN-P is alternately updated between the object weights and the pupil function weights.…”

Section: Introductionmentioning

confidence: 99%

Neural network model with positional deviation correction for Fourier ptychography

et al. 2021

View full text Add to dashboard Cite

Fourier ptychography is a new type of computational imaging technology, which uses a stack of low-resolution images obtained from overlapped apertures (or equivalent) to reconstruct super-resolved image. However, the accuracy of the aperture position will directly affect the quality and resolution of the reconstructed image. This paper proposes a new perspective for FP positional deviations correction using the neural network. We construct a trainable neural network to perform positional deviations correction along with object reconstruction. The real part and imaginary part of the object as well as the different and irregular positional deviations of each aperture are set as the weights of convolutional layer. The gradients over these weights are computed automatically, and gradient-based optimization algorithm is employed to recover the object and find the correct aperture position. Simulation and experiment are performed to verify our algorithm. The results show that the proposed algorithm can accurately find the aperture position and improve the reconstruction quality.

show abstract

Neural network model combined with pupil recovery for Fourier ptychographic microscopy

Cited by 28 publications

References 35 publications

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

Neural network model with positional deviation correction for Fourier ptychography

Contact Info

Product

Resources

About