Incorporating the image formation process into deep learning improves network performance in deconvolution applications

Li, Yue; Su, Yijun; Guo, Min; Han, Xiaofei; Liu, Shiyuan; Vishwasrao, Harshad D.; Li, Xuesong; Christensen, Ryan; Sengupta, Titas; Moyle, Mark W.; Chen, Jiji; Usdin, Ted B.; Colón-Ramos, Daniel A.; Liu, Huafeng; Wu, Yicong; Shroff, Hari

doi:10.1101/2022.03.05.483139

Cited by 5 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, we assess the performance of the Deep Learning (DL) based algorithm which we developed to assist in our mesoscale structural evaluation of the tissue boundary. DL is making its impact in various aspects of microscopy such as deconvolution 32 , super-resolution image generation [33][34][35][36] , classification 37,38 and segmentation 39,40 . Our DL based classification model is able to distinguish the informative volumes from the non-informative volumes by generating a probability map of non-empty volumes with high accuracy (Fig.…”

Section: Resultsmentioning

confidence: 99%

Signal Improved ultra-Fast Light-sheet Microscope (SIFT) for large tissue imaging

Prince

Garcia

Henn

et al. 2023

Preprint

View full text Add to dashboard Cite

Light-sheet fluorescence microscopy (LSFM) in conjunction with tissue clearing techniques enables morphological investigation of large tissues faster and with excellent optical sectioning. Recently, cleared tissue axially swept light-sheet microscope (ctASLM) demonstrated three-dimensional isotropic resolution in millimeter-scaled tissues. But ASLM based microscopes suffer from low detection signal and slow imaging speed. Here we report a simple and efficient imaging platform that employs precise control of two fixed distant light-sheet foci to carry out ASLM. This allowed us to carry out full field of view (FOV) imaging at 40 frames per second (fps) which is a four-fold improvement compared to the current state-of-the-art. In addition, in a particular frame rate, our method doubles the signal compared to the current ASLM technique. To augment the overall imaging performance, we also developed a deep learning based tissue information classifier that enables faster determination of tissue boundary. We demonstrated the performance of our imaging pipeline on various cleared tissue samples and demonstrated its robustness over a wide range of clearing protocols.

show abstract

Section: Resultsmentioning

confidence: 99%

Signal Improved ultra-Fast Light-sheet Microscope (SIFT) for large tissue imaging

Prince

Garcia

Henn

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast, deterministic algorithms such as MIRO are readily accessible to diverse image data formats, can maintain the quantitative details of the signal, and become especially preferable for discovering biological knowledge beyond available training datasets (64). For this reason, hybrid strategies have lately been formulated to exploit the advantages of both worlds (65)(66)(67). In particular, it has been proposed that the shearlet transform of vast curvilinear biological features can effectively reduce the complexity of the learning problems, thus permitting more efficient convergence for network training (68).…”

Section: Discussionmentioning

confidence: 99%

Optimal sparsity allows reliable system-aware restoration of fluorescence microscopy images

Mandracchia,

Liu,

Hua

et al. 2023

Sci. Adv.

View full text Add to dashboard Cite

Fluorescence microscopy is one of the most indispensable and informative driving forces for biological research, but the extent of observable biological phenomena is essentially determined by the content and quality of the acquired images. To address the different noise sources that can degrade these images, we introduce an algorithm for multiscale image restoration through optimally sparse representation (MIRO). MIRO is a deterministic framework that models the acquisition process and uses pixelwise noise correction to improve image quality. Our study demonstrates that this approach yields a remarkable restoration of the fluorescence signal for a wide range of microscopy systems, regardless of the detector used (e.g., electron-multiplying charge-coupled device, scientific complementary metal-oxide semiconductor, or photomultiplier tube). MIRO improves current imaging capabilities, enabling fast, low-light optical microscopy, accurate image analysis, and robust machine intelligence when integrated with deep neural networks. This expands the range of biological knowledge that can be obtained from fluorescence microscopy.

show abstract

“…Once trained, application of the networks is faster, requiring ∼5 minutes per volume (each 500 × 500 × 80 voxels). Given continued development in the rapidly growing field of deep learning, improved networks with fewer parameters 54 are likely to significantly shrink these times in the future. In considering our multi-step approach, we maintained as network input the 15 raw image volumes required for traditional 3D SIM reconstruction.…”

Section: Discussionmentioning

confidence: 99%

Three-dimensional structured illumination microscopy with enhanced axial resolution

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present two distinct, complementary methods for improving axial resolution in three-dimensional structured illumination microscopy (3D SIM) with minimal or no modification to the optical system. First, we show that placing a mirror directly opposite the sample enables 4-beam interference with higher spatial frequency content than 3D SIM illumination, offering near-isotropic imaging with ~120 nm lateral and 160 nm axial resolution. Second, we develop an improved deep learning method that can be directly applied to 3D SIM data, obviating the need for additional hardware. This procedure results in ~120 nm isotropic resolution and can be combined with denoising to facilitate volumetric imaging spanning dozens of time points. We demonstrate the potential of these advances by imaging a variety of cellular samples, delineating the nanoscale distribution of vimentin and microtubule filaments, observing the relative positions of caveolar coat proteins and lysosomal markers, and visualizing rich cytoskeletal dynamics within T-cells in the early stages of immune synapse formation.

show abstract

Incorporating the image formation process into deep learning improves network performance in deconvolution applications

Cited by 5 publications

References 36 publications

Signal Improved ultra-Fast Light-sheet Microscope (SIFT) for large tissue imaging

Signal Improved ultra-Fast Light-sheet Microscope (SIFT) for large tissue imaging

Optimal sparsity allows reliable system-aware restoration of fluorescence microscopy images

Three-dimensional structured illumination microscopy with enhanced axial resolution

Contact Info

Product

Resources

About