Holo-UNet: hologram-to-hologram neural network restoration for high fidelity low light quantitative phase imaging of live cells

Zhang, Zhiduo; Zheng, Yujie; Xu, Tianhua; Upadhya, Avinash; Lim, Yean Jin; Mathews, A. P.; Xie, Lexing; Lee, Woei Ming

doi:10.1364/boe.395302

Cited by 25 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CNN architecture is based on the encoder-decoder idea of UNet architecture 52 . Although initially, this architecture was used for a segmentation problem, recent studies have shown that this architecture is very efficient for the required restoration 16 , 53 , 54 . However, several changes have been made to the original architecture.…”

Section: Simulation and Experimentsmentioning

confidence: 99%

Imaging through diffuse media using multi-mode vortex beams and deep learning

Balasubramaniam

Biton

Arnon

2022

Sci Rep

View full text Add to dashboard Cite

Optical imaging through diffuse media is a challenging issue and has attracted applications in many fields such as biomedical imaging, non-destructive testing, and computer-assisted surgery. However, light interaction with diffuse media leads to multiple scattering of the photons in the angular and spatial domain, severely degrading the image reconstruction process. In this article, a novel method to image through diffuse media using multiple modes of vortex beams and a new deep learning network named “LGDiffNet” is derived. A proof-of-concept numerical simulation is conducted using this method, and the results are experimentally verified. In this technique, the multiple modes of Gaussian and Laguerre-Gaussian beams illuminate the displayed digits dataset number, and the beams are then propagated through the diffuser before being captured on the beam profiler. Furthermore, we investigated whether imaging through diffuse media using multiple modes of vortex beams instead of Gaussian beams improves the imaging system's imaging capability and enhances the network's reconstruction ability. Our results show that illuminating the diffuser using vortex beams and employing the “LGDiffNet” network provides enhanced image reconstruction compared to existing modalities. An enhancement of ~ 1 dB, in terms of PSNR, is achieved using this method when a highly scattering diffuser of grit 220 and width 2 mm (7.11 times the mean free path) is used. No additional optimizations or reference beams were used in the imaging system, revealing the robustness of the “LGDiffNet” network and the adaptability of the imaging system for practical applications in medical imaging.

show abstract

Section: Simulation and Experimentsmentioning

confidence: 99%

Imaging through diffuse media using multi-mode vortex beams and deep learning

Balasubramaniam

Biton

Arnon

2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In this approach the algorithm's input-output relationship is found during the neural network learning process and there is no need for its a priori analytical definition. In general, in the case of image analysis the most suitable and flourishing type of neural networks are CNNs [62][63][64][65][66][67][68][69][70][71][72][74][75][76][77][78][79][80][81][82][83], where the basic mathematical operation in each neuron is defined by the convolution.…”

Section: Fully Automatic and Accelerated Vid Dedicated To Fringe Patt...mentioning

confidence: 99%

“…Neural network-based solutions were already applied to support fringe pattern analysis on different stages, e.g. prefiltration [61][62][63][64][65][66][67], optimization of the window parameters for Fourier transform [68], phase estimation [69][70][71][72][73][74][75], phase unwrapping [76][77][78][79][80][81][82] and local fringe density map estimation [83]. Nevertheless, in this paper deliberately another approach was chosen than to directly determine the fringe pattern background filtration result by the neural network as an alternative to the solutions proposed in [66,67].…”

Section: Introductionmentioning

confidence: 99%

DeepVID: deep-learning accelerated variational image decomposition model tailored to fringe pattern filtration

Cywińska

Szumigaj

Kołodziej

et al. 2023

J. Opt.

View full text Add to dashboard Cite

The algorithms defined as variational image decomposition (VID) constitute the state-of-the-art in solving the image prefiltration problem. However, the discussion about the advantages and disadvantages of different variational image decomposition models in the context of fringe pattern prefiltration is yet to be addressed and this work is the first one leaning into this issue. The unsupervised variational image decomposition (uVID) algorithm allows for automatic, accurate and robust preprocessing of diverse fringe patterns and introduces the parameters and stopping criterion for Chambolle’s iterative projection algorithm to separate the fringes and background. However, determining the stopping criterion in each iteration is a severely time-consuming process, which is particularly important given the fact that in many cases thousands of iterations must be calculated to obtain a satisfactory fringe pattern decomposition result. Therefore, the idea of using convolutional neural network to map the relationship between the fringe pattern spatial intensity distribution and the required number of Chambolle projection iterations has emerged. That way, it is no longer required to determine the value of the stopping criterion in every iteration, but the appropriate number of iterations is known in advance via machine learning process. We showed that the calculation time is reduced on average by 3-4 times by employing the deep learning-based acceleration (DeepVID) without jeopardizing the overall accuracy of the prefiltration. This way an important progress in developing uVID algorithm features towards real-time studies of dynamic phenomena is reported in this contribution. For the sake of metrological figure of merit, we employ deep learning based solution, for the first time to the best of our knowledge, to accelerate powerful and well-established VID approach, not to bypass it completely.

show abstract

“…3, a U-net model consists of total 19 convolutional layers for feature extraction, with four maximum pooling (max-pool) layers, and four transpose convolutional layers to generate multi-scale features. The number of convolution layers in each scale was chosen from the empirical results following the previous works [28,[30][31][32][33]. The kernel size of each convolutional layer including transpose convolution is chosen as 3×3.…”

Section: A U-net Modelingmentioning

confidence: 99%

Patch-Based U-Net Model for Isotropic Quantitative Differential Phase Contrast Imaging

Vyas

Lin

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Quantitative differentialphase-contrast (qDPC) imaging is a label-free phase retrieval method for weak phase objects using asymmetric illumination. However, qDPC imaging with fewer intensity measurements leads to anisotropic phase distribution in reconstructed images. In order to obtain isotropic phase transfer function, multiple measurements are required; thus, it is a time-consuming process. Here, we propose the feasibility of using deep learning (DL) method for isotropic qDPC microscopy from the least number of measurements. We utilize a commonly used convolutional neural network namely U-net architecture, trained to generate 12-axis isotropic reconstructed cell images (i.e. output) from 1-axis anisotropic cell images (i.e. input). To further extend the number of images for training, the U-net model is trained with a patch-wise approach. In this work, seven different types of living cell images were used for training, validation, and testing datasets. The results obtained from testing datasets show that our proposed DL-based method generates 1-axis qDPC images of similar accuracy to 12axis measurements. The quantitative phase value in the region of interest is recovered from 66% up to 97%, compared to ground-truth values, providing solid evidence for improved phase uniformity, as well as retrieved missing spatial frequencies in 1-axis reconstructed images. In addition, results from our model are compared with paired and unpaired CycleGANs. Higher PSNR and SSIM values show the advantage of using the U-net model for isotropic qDPC microscopy. The proposed DL-based method may help in performing high-resolution quantitative studies for cell biology.

show abstract

Holo-UNet: hologram-to-hologram neural network restoration for high fidelity low light quantitative phase imaging of live cells

Cited by 25 publications

References 18 publications

Imaging through diffuse media using multi-mode vortex beams and deep learning

Imaging through diffuse media using multi-mode vortex beams and deep learning

DeepVID: deep-learning accelerated variational image decomposition model tailored to fringe pattern filtration

Patch-Based U-Net Model for Isotropic Quantitative Differential Phase Contrast Imaging

Contact Info

Product

Resources

About