High-resolution limited-angle phase tomography of dense layered objects using deep neural networks

Goy, Alexandre; Rughoobur, Girish; Li, Shuai; Arthur, Kwabena; Akinwande, Akintunde I.; Barbastathis, George

doi:10.1073/pnas.1821378116

Cited by 63 publications

(74 citation statements)

References 65 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1). Unlike other recent works that propose to use supervised deep learning to aid in computational image reconstruction problems [35][36][37][38][39], including a number of works that rely on multi-angle illumination [30][31][32][40][41][42][43], the technique proposed here does not require any pre-training or dataset-specific assumptions. Instead, during iterative object reconstruction, DP-DT simply performs its optimization updates with respect to the parameters of a CNN, as opposed to directly updating the object voxels.…”

Section: Introductionmentioning

confidence: 99%

Diffraction tomography with a deep image prior

Zhou¹,

Horstmeyer²

2020

Opt. Express

View full text Add to dashboard Cite

We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy.

show abstract

Section: Introductionmentioning

confidence: 99%

Diffraction tomography with a deep image prior

Zhou¹,

Horstmeyer²

2020

Opt. Express

View full text Add to dashboard Cite

show abstract

“…We modeled the Bessel beam as a constant line segment for computational simplicity, but models of the actual point spread function could be used for computing the inverse Radon transform. Approaches combining imaging physics and machine learning have been successfully applied in other tomography techniques for improving the reconstruction from sparse, shallow angle projections [13,[16][17][18][19][20]. Our networks were limited by GPU memory and therefore only low resolution images (128 pixel resolution instead of 512) were used.…”

Section: Discussionmentioning

confidence: 99%

“…Volume information is obtained from four independent projections recorded from four different angles using temporally multiplexed, tilted Bessel beams in a single frame scan. For volume reconstruction we combine inverse Radon transforms adapted for Bessel beam scanning with machine learning [13,[16][17][18][19][20]. Machine learning has been shown for example in optical phase imaging to allow high resolution reconstruction from sparse projections at shallow angles similar to the ones used here [18].…”

Section: Introductionmentioning

confidence: 99%

“…For volume reconstruction we combine inverse Radon transforms adapted for Bessel beam scanning with machine learning [13,[16][17][18][19][20]. Machine learning has been shown for example in optical phase imaging to allow high resolution reconstruction from sparse projections at shallow angles similar to the ones used here [18]. We show that three-dimensional convolutional neural networks (U-net) trained on suitable datasets similarly improve reconstructions in Bessel beam tomography.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bessel beam tomography for fast volume imaging

Valle

Seelig

2019

Preprint

View full text Add to dashboard Cite

Light microscopy on dynamic samples, for example neural activity in the brain, requires imaging large volumes at high rates. Here, we develop a tomography approach for scanning fluorescence microscopy which allows recording volume images at frame scan rates. Volumes are imaged by simultaneously recording four independent projections at different angles using temporally multiplexed, tilted Bessel beams. From the resulting projections, volumes are reconstructed using inverse Radon transforms combined with three dimensional convolutional neural networks (U-net). This tomography approach is suitable for experiments requiring fast volume imaging of sparse samples, as for example often encountered when imaging neural activity in the brain.

show abstract

“…An interesting alternative method for the inverse problem, also nonlinear, of reconstructing the three-dimensional (3D) refractive index distribution from intensity projections, is to define the DNN architecture according to the strong scattering model and store the refractive index values as weights of the DNN after training [44]. This "index-storing" DNN itself was subsequently used as Approximant to a traditional DNN for improving the estimates in 3D distributions with exceptionally small and highcontrast features or when the range of available angles of projec-tion is severely limited [45]. Extensive reviews of deep learning use for inverse problems can be found in [36,46,47].…”

Section: Introductionmentioning

confidence: 99%

Learning to synthesize: robust phase retrieval at low photon counts

Deng

Goy³

et al. 2020

Light Sci Appl

Self Cite

View full text Add to dashboard Cite

The quality of inverse problem solutions obtained through deep learning [Barbastathis et al, 2019] is limited by the nature of the priors learned from examples presented during the training phase. In the case of quantitative phase retrieval [Sinha et al 2017, Goy et al 2019], in particular, spatial frequencies that are underrepresented in the training database, most often at the high band, tend to be suppressed in the reconstruction. Ad hoc solutions have been proposed, such as pre-amplifying the high spatial frequencies in the examples [Li et al, 2018]; however, while that strategy improves resolution, it also leads to high-frequency artifacts as well as lowfrequency distortions in the reconstructions. Here, we present a new approach that learns separately how to handle the two frequency bands, low and high; and also learns how to synthesize these two bands into the full-band reconstructions. We show that this "learning to synthesize" (LS) method yields phase reconstructions of high spatial resolution and artifact-free; and it is also resilient to high-noise conditions, e.g. in the case of very low photon flux. In addition to the problem of quantitative phase retrieval, the LS method is applicable, in principle, to any inverse problem where the forward operator treats different frequency bands unevenly, i.e. is ill-posed.

show abstract

High-resolution limited-angle phase tomography of dense layered objects using deep neural networks

Cited by 63 publications

References 65 publications

Diffraction tomography with a deep image prior

Diffraction tomography with a deep image prior

Bessel beam tomography for fast volume imaging

Learning to synthesize: robust phase retrieval at low photon counts

Contact Info

Product

Resources

About