Fourier ptychographic microscopy reconstruction with multiscale deep residual network

Zhang, Jizhou; Xu, Tingfa; Shen, Ziyi; Qiao, Yifan; Zhang, Yizhou

doi:10.1364/oe.27.008612

Cited by 61 publications

(33 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is natural to introduce the idea of CNN into this inverse problem and learn an underlying mapping from the low-resolution input to a high-resolution output. [24][25][26][27] However, there exist some specific issues in biomedical applications. First, different from natural image applications, it is hard for biomedical imaging problems to access large amounts of images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

et al. 2021

View full text Add to dashboard Cite

Significance: Fourier ptychography (FP) is a computational imaging approach that achieves high-resolution reconstruction. Inspired by neural networks, many deep-learning-based methods are proposed to solve FP problems. However, the performance of FP still suffers from optical aberration, which needs to be considered. Aim:We present a neural network model for FP reconstructions that can make proper estimation toward aberration and achieve artifact-free reconstruction.Approach: Inspired by the iterative reconstruction of FP, we design a neural network model that mimics the forward imaging process of FP via TensorFlow. The sample and aberration are considered as learnable weights and optimized through back-propagation. Especially, we employ the Zernike terms instead of aberration to decrease the optimization freedom of pupil recovery and perform a high-accuracy estimation. Owing to the auto-differentiation capabilities of the neural network, we additionally utilize total variation regularization to improve the visual quality.Results: We validate the performance of the reported method via both simulation and experiment. Our method exhibits higher robustness against sophisticated optical aberrations and achieves better image quality by reducing artifacts. Conclusions:The forward neural network model can jointly recover the high-resolution sample and optical aberration in iterative FP reconstruction. We hope our method that can provide a neural-network perspective to solve iterative-based coherent or incoherent imaging problems.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Once the system setup is changed or the system aberration is introduced, the performance of the deep-learning-based network will be degraded. Toward the first dilemma, Zhang et al 27 generated datasets with simulations to train the network. However, there is a lack of related networks to solve the second issue.…”

Section: Introductionmentioning

confidence: 99%

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In recent years, with the rapid development of the deep learning (DL) technique, algorithms based on deep convolutional neural network (DCNN) have been proposed to solve many image processing problems, such as image de‐noising [22], single image super‐resolution [23–25] and phase retrieval [26, 29]. Since the purpose of FPM is to synthesize a high‐resolution complex field from multiple low‐resolution images, several algorithms have employed DCNN to solve the FPM problem [27–32] which greatly improve the reconstruction speed. However, in Reference [27] they need traditional method to generate a preliminary result as the input of network, and then the network is trained to optimize the input instead of using the low‐resolution images to reconstruct, and in Reference [28] the performance of the network is mainly demonstrated by simulation and lack the description of the generalization to the actual experimental dataset.…”

Section: Introductionmentioning

confidence: 99%

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Sun

Shao

Zhu

et al. 2021

Journal of Biophotonics

View full text Add to dashboard Cite

Fourier ptychographic microscopy is a promising imaging technique which can circumvent the space‐bandwidth product of the system and achieve a reconstruction result with wide field‐of‐view (FOV), high‐resolution and quantitative phase information. However, traditional iterative‐based methods typically require multiple times to get convergence, and due to the wave vector deviation in different areas, the millimeter‐level full‐FOV cannot be well reconstructed once and typically required to be separated into several portions with sufficient overlaps and reconstructed separately, which makes traditional methods suffer from long reconstruction time for a large‐FOV (of the order of minutes) and limits the application in real‐time large‐FOV monitoring of live sample in vitro. Here we propose a novel deep‐learning based method called DFNN which can be used in place of traditional iterative‐based methods to increase the quality of single large‐FOV reconstruction and reducing the processing time from 167.5 to 0.1125 second. In addition, we demonstrate that by training based on the simulation dataset with high‐entropy property (Opt. Express 28, 24 152 [2020]), DFNN could has fine generalizability and little dependence on the morphological features of samples. The superior robustness of DFNN against noise is also demonstrated in both simulation and experiment. Furthermore, our model shows more robustness against the wave vector deviation. Therefore, we could achieve better results at the edge areas of a single large‐FOV reconstruction. Our method demonstrates a promising way to perform real‐time single large‐FOV reconstructions and provides further possibilities for real‐time large‐FOV monitoring of live samples with sub‐cellular resolution.

show abstract

“…We also build a deep neural network based on generative adversarial network (GAN) [31]- [33] framework to transform the input data into the desired output image. Recently, researches have proposed several methods that train neural networks with large scale datasets and perform better FPM reconstruction with neural networks [34]- [37]. The main difference between MASRM and these methods is that we collect real data not FPM reconstruction results as ground truth.…”

Section: Introductionmentioning

confidence: 99%

High-Throughput Deep Learning Microscopy Using Multi-Angle Super-Resolution

Zhang

et al. 2020

IEEE Photonics J.

Self Cite

View full text Add to dashboard Cite

Biomedical applications such as pathology and hematology expect microscopes with high space-bandwidth product (SBP) which is difficult to achieve with conventional microscope setup. By applying a deep neural network, we demonstrate a high spacebandwidth product microscopic technique termed multi-angle super-resolution microscopy (MASRM) to achieve high-resolution imaging with the low-magnification objective. We design a multiple-branch deep residual network which extracts high-frequency information and color information in obliquely-illuminated low-resolution input images and generates high-resolution output. To train our network, we build a well-registered dataset in which both low-resolution input and high-resolution target are real captured images. We carry out detailed experiments to demonstrate the effectiveness of MASRM and compare it with a computational imaging technique termed Fourier ptychographic microscopy (FPM). This data-driven technique unleashes the potential of traditional microscopes with low cost and has broad prospects in biomedical applications.

show abstract

Fourier ptychographic microscopy reconstruction with multiscale deep residual network

Cited by 61 publications

References 36 publications

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

Double‐flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

High-Throughput Deep Learning Microscopy Using Multi-Angle Super-Resolution

Contact Info

Product

Resources

About