Neural network model assisted Fourier ptychography with Zernike aberration recovery and total variation constraint

Abstract

Significance: Fourier ptychography (FP) is a computational imaging approach that achieves high-resolution reconstruction. Inspired by neural networks, many deep-learning-based methods are proposed to solve FP problems. However, the performance of FP still suffers from optical aberration, which needs to be considered. Aim:We present a neural network model for FP reconstructions that can make proper estimation toward aberration and achieve artifact-free reconstruction.Approach: Inspired by the iterative reconstruction of FP, we design a neural network model that mimics the forward imaging process of FP via TensorFlow. The sample and aberration are considered as learnable weights and optimized through back-propagation. Especially, we employ the Zernike terms instead of aberration to decrease the optimization freedom of pupil recovery and perform a high-accuracy estimation. Owing to the auto-differentiation capabilities of the neural network, we additionally utilize total variation regularization to improve the visual quality.Results: We validate the performance of the reported method via both simulation and experiment. Our method exhibits higher robustness against sophisticated optical aberrations and achieves better image quality by reducing artifacts. Conclusions:The forward neural network model can jointly recover the high-resolution sample and optical aberration in iterative FP reconstruction. We hope our method that can provide a neural-network perspective to solve iterative-based coherent or incoherent imaging problems.

Deep Multi-Feature Transfer Network for Fourier Ptychographic Microscopy Imaging Reconstruction

“…The DMFTN reconstruction method proposed in this paper is verified on the simulation and experimental data sets. Using the simulation experimental data, the reconstruction results of the FPM reconstruction method under deep learning are compared and evaluated with the iterative phase recovery reconstruction alternating projection G-S [ 34 ] method, the latest Zhang [ 19 ] and the Zuo’s AS [ 39 ] method.…”

“…Jiang et al [ 18 ] modeled FPM through convolutional neural network and reconstructed FPM through back propagation. Zhang et al [ 19 ] introduced Zernike aberration recovery and total variation constraints based on neural network to ensure the quality of FPM reconstruction and at the same time to correct aberration. The above work is still based on iterative algorithms and does not give full play to the advantages of deep learning.…”

Physics‐based learning with channel attention for Fourier ptychographic microscopy

“…Since each image can be estimated independently, no training dataset is required. The physics‐based learning approach is first introduced into FPM by Jiang et al [36] and further improved by Sun et al [37–39, 45]. The physics‐based learning approach succeeds in combining the interpretability of the physical model and the automatic differentiation of networks.…”

“…The basic method used to perform FPM reconstruction is the alternate projection (AP) that iteratively estimates low‐resolution complex fields and updates the corresponding spectrum regions. Recently, methods have been proposed that solving the FPM reconstruction problem with physics‐based neural networks (PbNNs) [36–39]. These methods treat the real and imaginary parts of the sample spectrum as trainable parameters and model the FPM reconstruction process with feed‐forward neural networks.…”

A Fourier Ptychographic Microscopy Reconstruction Method Based on SwinIR Physical Model

“…In terms of FPM reconstruction, traditional reconstruction algorithms such as G-S (Gerchberg-Saxton) [2] and A-S (adaptive step-size) [3] have a fast reconstruction speed, but the effect is somewhat unsatisfactory. In the field of deep learning, researchers proposed INNM [4], DFNN [5], and other reconstruction methods based on deep learning. Although these reconstruction methods have a good reconstruction effect, it is difficult for the public to bear high equipment costs because they require a large number of data for training and require high graphics cards to support the experimental part.…”