Cross-Modality Deep Learning Achieves Super-Resolution in Fluorescence Microscopy

Wang, Hongda; Rivenson, Yair; Jin, Yiyin; Wei, Zhensong; Gao, Ronald; Günaydın, Harun; Bentolila, Laurent A.; Kural, Cömert; Özcan, Aydogan

doi:10.1364/cleo_si.2019.stu4h.2

Cited by 7 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These data repositories can then be used in strategies to improve predictive performance of deep-learning models, by iteratively re-training and fine-tunning predictive models using constantly improving datasets, such as "human-in-the-loop" approaches that apply active learning methods (Greenwald et al, 2022;Pachitariu & Stringer, 2022). Similarly, sophisticated transfer learning (Jin et al, 2020;Shyam & Selvam, 2022;von Chamier et al, 2021;H. Wang et al, 2019) applications could be implemented in this setting, e.g.…”

Section: Discussionmentioning

confidence: 99%

nf-root: a best-practice pipeline for deep learning-based analysis of apoplastic pH in microscopy images of developmental zones in plant root tissue

Wanner

Cuellar²,

Rausch³

et al. 2023

Preprint

View full text Add to dashboard Cite

Here we reportnextflow-root(nf-root), a novel best-practice pipeline for deep learning-based analysis of fluorescence microscopy images of plant root tissue, aimed at studying hormonal mechanisms associated with cell elongation, given the vital role that plant hormones play in the development and growth of plants. This bioinformatics pipeline performs automatic identification of developmental zones in root tissue images, and analysis of apoplastic pH measurements of tissue zones, which is useful for modeling plant hormone signaling and cell physiological responses. Mathematical models of physiological responses of plant hormones, such as brassinolide, have been successfully established for certain root tissue types, by evaluating apoplastic pH via fluorescence imaging. However, the generation of data for this modeling is time-consuming, as it requires the manual segmentation of tissue zones and evaluation of large amounts of microscopy data. We introduce a high throughput, highly reproducibleNextflowpipeline based onnf-corestandards that automates tissue zone segmentation by implementing a deep-learning module, which deploys deterministically trained (i.e. bit-exact reproducible) convolutional neural network models, and augments the segmentation predictions with measures of predictionuncertaintyand modelinterpretability, aiming to facilitate result interpretation and verification by experienced plant biologists. To train our segmentation prediction models, we created a publicly available dataset composed of confocal microscopy images ofA. thalianaroot tissue using the pH-sensitive fluorescence indicator, and manually annotated segmentation masks that identify relevant tissue zones. We applied this pipeline to analyze exemplary data, and observed a high statistical similarity between the manually generated results and the output ofnf-root. Our results indicate that this approach achieves near human-level performance, and significantly reduces the time required to analyze large volumes of data, from several days to hours.

show abstract

Section: Discussionmentioning

confidence: 99%

nf-root: a best-practice pipeline for deep learning-based analysis of apoplastic pH in microscopy images of developmental zones in plant root tissue

Wanner

Cuellar²,

Rausch³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Currently, the most commonly used devices for the high-resolution imaging of biological or biomedical targets include confocal microscopes [1], stimulated emission depletion (STED) microscopes [2], and structured light illumination microscopes (SIM) [3] etc. Furthermore, many algorithms have been developed to improve the spatial resolution and signal-to-noise ratio (SNR) of biological images, including degenerate-model-based algorithms (e.g., deconvolution [4][5][6][7][8]), mathematical transformation-based algorithms (e.g., spectrum analysis [9,10], DWT analysis [11][12][13][14][15][16]), and machine-learning-based algorithms (e.g., deep learning [17][18][19]). However, most of these algorithms focus on a single task, e.g., inhibiting noise, identifying structure contours, or improving resolution.…”

Section: Introduction 1research Backgroundmentioning

confidence: 99%

Feature Extraction of 3T3 Fibroblast Microtubule Based on Discrete Wavelet Transform and Lucy–Richardson Deconvolution Methods

et al. 2022

View full text Add to dashboard Cite

Accompanied by the increasing requirements of the probing micro/nanoscopic structures of biological samples, various image-processing algorithms have been developed for visualization or to facilitate data analysis. However, it remains challenging to enhance both the signal-to-noise ratio and image resolution using a single algorithm. In this investigation, we propose a composite image processing method by combining discrete wavelet transform (DWT) and the Lucy–Richardson (LR) deconvolution method, termed the DWDC method. Our results demonstrate that the signal-to-noise ratio and resolution of live cells’ microtubule networks are considerably improved, allowing the recognition of features as small as 120 nm. The method shows robustness in processing the high-noise images of filament-like biological structures, e.g., the cytoskeleton networks captured by fluorescent microscopes.

show abstract

“…A similar approach was performed for translating quantitative phase imaging into three different stains, namely H&E stain, Jone’s stain, and Masson’s trichrome stain [ 26 ]. CGANs were also employed to increase the spatial resolution [ 27 , 28 ] and remove speckle noise from optical microscopic images. Similarly, the cycle CGAN were utilized to stain a H&E stained image into an immunohistochemically (IHC) stained image.…”

Section: Introductionmentioning

confidence: 99%

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Pradhan

Meyer

Stallmach

et al. 2021

Biomed. Opt. Express

View full text Add to dashboard Cite

Hematoxylin and Eosin (H&E) staining is the ’gold-standard’ method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for ’real-time’ disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author’s best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.

show abstract

Cross-Modality Deep Learning Achieves Super-Resolution in Fluorescence Microscopy

Cited by 7 publications

References 1 publication

nf-root: a best-practice pipeline for deep learning-based analysis of apoplastic pH in microscopy images of developmental zones in plant root tissue

nf-root: a best-practice pipeline for deep learning-based analysis of apoplastic pH in microscopy images of developmental zones in plant root tissue

Feature Extraction of 3T3 Fibroblast Microtubule Based on Discrete Wavelet Transform and Lucy–Richardson Deconvolution Methods

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Contact Info

Product

Resources

About