Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Mahmood, Faisal; Borders, Daniel; Chen, Richard; McKay, Gregory N.; Salimian, Kevan J.; Baras, Alexander S.; Durr, Nicholas J.

doi:10.48550/arxiv.1810.00236

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applying stain-to-stain translation to pathological tissues presents a barrier against wide adoption of the practice because of organ and disease specific cross segmentation loss Mahmood et al [2018]. We address this problem by demonstrating how cluster specific markers can improve loss on multichannel predictions.…”

Section: Discussionmentioning

confidence: 99%

A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels

Saurav¹,

Nasr²,

Paul³

et al. 2022

Preprint

View full text Add to dashboard Cite

Histopathological work in clinical labs relying on immunostaining of proteins represents a bottleneck in processing medical tissue samples. Multiplexed spatial proteomics imaging can increase interpretive power but cannot cost-effectively sample the entire proteomic retinue important to diagnostic medicine or drug development. Here we present a structural similarity index measure (SSIM) guided conditional Generative Adversarial Network (cGAN) that generatively performs image-to-image (i2i) synthesis to generate photo-accurate protein channels in multiplexed spatial proteomics images. This approach can be utilized to accurately generate missing spatial proteomics channels that were not included during experimental data collection either at the bench or the clinic. Experimental spatial proteomic data from the Human BioMolecular Atlas Program (HuBMAP) was used to generate spatial representations of missing proteins through a U-Net based image synthesis pipeline. HuBMAP channels were hierarchically clustered by the (SSIM) as a heuristic to obtain the minimal set needed to recapitulate the underlying biology represented by the spatial landscape of proteins. We subsequently prove that our SSIM based architecture allows for scaling of generative image synthesis to slides with up to 100 channels, which is better than current state of the art algorithms which are limited to data with 11 channels. We validate these claims by generating a new experimental spatial proteomics data set from human lung adenocarcinoma tissue sections and show that a model trained on HuBMAP can accurately synthesize channels from our new data set. The ability to recapitulate experimental data from sparsely stained multiplexed histological slides containing spatial proteomic will have tremendous impact on medical diagnostics and drug development, and also raises important questions on the medical ethics of utilizing data produced by generative image synthesis in the clinical setting. The algorithm that we present in this paper will allow researchers and clinicians to save time and Preprint. Under review.

show abstract

Section: Discussionmentioning

confidence: 99%

A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels

Saurav¹,

Nasr²,

Paul³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Segmentation and classification are among the most popular techniques used in computer vision problems for object detection and interpretation for the majority of datasets [2,3]. In particular, cell segmentation is the most popular topic for biomedical image analysis with the plethora of literature on segmenting different cell types [4][5][6][7]. Some of the most used methods in cell segmentation literature are thresholding, the active contour model, the watershed algorithm, and the deep learning approach [8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Combined segmentation and classification-based approach to automated analysis of biomedical signals obtained from calcium imaging

et al. 2023

View full text Add to dashboard Cite

Automated screening systems in conjunction with machine learning-based methods are becoming an essential part of the healthcare systems for assisting in disease diagnosis. Moreover, manually annotating data and hand-crafting features for training purposes are impractical and time-consuming. We propose a segmentation and classification-based approach for assembling an automated screening system for the analysis of calcium imaging. The method was developed and verified using the effects of disease IgGs (from Amyotrophic Lateral Sclerosis patients) on calcium (Ca2+) homeostasis. From 33 imaging videos we analyzed, 21 belonged to the disease and 12 to the control experimental groups. The method consists of three main steps: projection, segmentation, and classification. The entire Ca2+ time-lapse image recordings (videos) were projected into a single image using different projection methods. Segmentation was performed by using a multi-level thresholding (MLT) step and the Regions of Interest (ROIs) that encompassed cell somas were detected. A mean value of the pixels within these boundaries was collected at each time point to obtain the Ca2+ traces (time-series). Finally, a new matrix called feature image was generated from those traces and used for assessing the classification accuracy of various classifiers (control vs. disease). The mean value of the segmentation F-score for all the data was above 0.80 throughout the tested threshold levels for all projection methods, namely maximum intensity, standard deviation, and standard deviation with linear scaling projection. Although the classification accuracy reached up to 90.14%, interestingly, we observed that achieving better scores in segmentation results did not necessarily correspond to an increase in classification performance. Our method takes the advantage of the multi-level thresholding and of a classification procedure based on the feature images, thus it does not have to rely on hand-crafted training parameters of each event. It thus provides a semi-autonomous tool for assessing segmentation parameters which allows for the best classification accuracy.

show abstract

“…Fu et al [5] used this model to transform artificial nuclei segmentation masks to synthetic nuclei fluorescence images, thus producing a large annotated dataset of nuclei required for training a nuclei segmentation CNN. Mahmood et al [6] developed a similar approach for generating training datasets for nuclei segmentation in hematoxylin and eosin stained (H&E) images. Additionally, they trained a CycleGAN to imitate nuclei segmentation that was performed by human experts.…”

Section: Introductionmentioning

confidence: 99%

Restoration of Marker Occluded Hematoxylin and Eosin Stained Whole Slide Histology Images Using Generative Adversarial Networks

Venkatesh¹,

Shaht

Chen³

et al. 2020

2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)

View full text Add to dashboard Cite

It is common for pathologists to annotate specific regions of the tissue, such as tumor, directly on the glass slide with markers. Although this practice was helpful prior to the advent of histology whole slide digitization, it often occludes important details which are increasingly relevant to immunooncology due to recent advancements in digital pathology imaging techniques. The current work uses a generative adversarial network with cycle loss to remove these annotations while still maintaining the underlying structure of the tissue by solving an image-to-image translation problem. We train our network on up to 300 whole slide images with marker inks and show that 70% of the corrected image patches are indistinguishable from originally uncontaminated image tissue to a human expert. This portion increases 97% when we replace the human expert with a deep residual network. We demonstrated the fidelity of the method to the original image by calculating the correlation between image gradient magnitudes. We observed a revival of up to 94,000 nuclei per slide in our dataset, the majority of which were located on tissue border.

show abstract

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Cited by 3 publications

References 0 publications

A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels

A SSIM Guided cGAN Architecture For Clinically Driven Generative Image Synthesis of Multiplexed Spatial Proteomics Channels

Combined segmentation and classification-based approach to automated analysis of biomedical signals obtained from calcium imaging

Restoration of Marker Occluded Hematoxylin and Eosin Stained Whole Slide Histology Images Using Generative Adversarial Networks

Contact Info

Product

Resources

About