Multi-modal digital pathology for colorectal cancer diagnosis by high-plex immunofluorescence imaging and traditional histology of the same tissue section

Lin, Jia-Ren; Yuan, Chen; Campton, Daniel; Cooper, Jeremy; Coy, Shannon; Yapp, Clarence; Tefft, Juliann B; McCarty, Erin; Ligon, Keith L.; Rodig, Scott J.; Reese, Steven; George, Tad; Santagata, Sandro; Sorger, Peter K.

doi:10.1101/2022.09.28.509927

Cited by 7 publications

(5 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Virtual staining (Burlingame et al, 2018; with reduced panels could improve opportunities for transition to the clinic. Moreover, technologies such as Orion from Rarecyte (Lin et al, 2022) can measure up to 16-18 markers in one cycle using spectral deconvolution, which can be combined with our approach to enabling even greater cell type differentiation, increasing the biomarker measurements from 20 to 32(⇡ 20 ⇥ 1.64) or more.…”

Section: Discussionmentioning

confidence: 99%

MIM-CyCIF: Masked Imaging Modeling for Enhancing Cyclic Immunofluorescence (CyCIF) with Panel Reduction and Imputation

Sims

Chang

2023

Preprint

View full text Add to dashboard Cite

Cyclic Immunofluorescence (CyCIF) has emerged as a powerful technique that can measure multiple biomarkers in a single tissue sample but it is limited in panel size due to technical issues and tissue loss. We develop a computational model that imputes a surrogate in silico high-plex CyCIF from only a few experimentally measured biomarkers by learning co-expression and morphological patterns at the single-cell level. The reduced panel is optimally designed to enable full reconstruction of an expanded marker panel that retains the information from the original panel necessary for downstream analysis. Using a masked image modeling approach based on the self-supervised training objective of reconstructing full images at the single-cell level, we demonstrate significant performance improvement over previous attempts on the breast cancer tissue microarray dataset. Our approach offers users access to a more extensive set of biomarkers beyond what has been experimentally measured. It also allows for allocating resources toward exploring novel biomarkers and facilitates greater cell type differentiation and disease characterization. Additionally, it can handle assay failures such as low-quality markers, technical noise, and/or tissue loss in later rounds as well as artificially upsample to include additional panel markers.

show abstract

Section: Discussionmentioning

confidence: 99%

MIM-CyCIF: Masked Imaging Modeling for Enhancing Cyclic Immunofluorescence (CyCIF) with Panel Reduction and Imputation

Sims

Chang

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Quantitative comparisons of real and virtual CyCIF stains exposed the challenge of using adjacent sections to train models, where image contents are subtly but appreciably different between sections at single-cell resolution. This challenge could be overcome in future studies by staining each tissue section first with CyCIF, then terminally with H&E 12,28 . That being said, this study and those like it take for granted that histology workflows are inherently destructive since serial sectioning and processing of tissue can preclude tissue from being used in other assays.…”

Section: Discussionmentioning

confidence: 99%

3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding

Burlingame

Ternes

Lin

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Tissue-based sampling and diagnosis are defined as the extraction of information from certain limited spaces and its diagnostic significance of a certain object. Pathologists deal with issues related to tumor heterogeneity since analyzing a single sample does not necessarily capture a representative depiction of cancer, and a tissue biopsy usually only presents a small fraction of the tumor. Many multiplex tissue imaging platforms (MTIs) make the assumption that tissue microarrays (TMAs) containing small core samples of 2-dimensional (2D) tissue sections are a good approximation of bulk tumors although tumors are not 2D. However, emerging whole slide imaging (WSI) or 3D tumor atlases that employ MTIs like cyclic immunofluorescence (CyCIF) strongly challenge this assumption. In spite of the additional insight gathered by measuring the tumor microenvironment in WSI or 3D, it can be prohibitively expensive and time-consuming to process tens or hundreds of tissue sections with CyCIF. Even when resources are not limited, the criteria for region-of-interest (ROI) selection in tissues for downstream analysis remain largely qualitative and subjective as stratified sampling requires the knowledge of objects and evaluates their features. Despite the fact TMAs fail to adequately approximate whole tissue features, a theoretical subsampling of tissue exists that can best represent the tumor in the whole slide image. To address these challenges, we propose deep learning approaches to learn multi-modal image translation tasks from two aspects: 1) generative modeling approach to reconstruct 3D CyCIF representation and 2) co-embedding CyCIF image and Hematoxylin and Eosin (H&E) section to learn multi-modal mappings by a cross-domain translation for minimum representative ROI selection. We demonstrate that generative modeling enables a 3D virtual CyCIF reconstruction of a colorectal cancer specimen given a small subset of the imaging data at training time. By co-embedding histology and MTI features, we propose a simple convex optimization for objective ROI selection. We demonstrate the potential application of ROI selection and the efficiency of its performance with respect to cellular heterogeneity.

show abstract

“…Slides were then rinsed again with surfactant wash buffer and placed in a humidified stain tray and incubated with SYTOX Blue (Thermo Fisher), ArgoFluor™ 845 mouse-anti-DIG in PBS-Antibody Stabilizer containing 10% goat serum for 30 minutes at room temperature. The slides were then rinsed a final time with surfactant wash buffer and PBS, coverslipped with ArgoFluor™ Mounting Media (RareCyte, Inc.) and dried overnight (Lin et al, 2022).…”

Section: Methodsmentioning

confidence: 99%

High-dimensional single-cell definition of CLL T cells identifies Galectin-9 as novel immunotherapy target

Cid

Wong

Botana

et al. 2022

Preprint

View full text Add to dashboard Cite

Failure of immunotherapy after applying checkpoint inhibitors or CAR-T cells is linked to T cell exhaustion. Here, we explored the T cell landscape in chronic lymphocytic leukemia (CLL) by single-cell omics analyses of blood, bone marrow and lymph node samples of patients and spleen samples of a CLL mouse model. By single-cell RNA-sequencing, mass cytometry (CyTOF), and multiplex image analysis of tissue microarrays, we defined the spectrum of phenotypes and transcriptional programs of T cells and and their differentiation state trajectories. We identified disease-specific accumulation of distinct regulatory T cell subsets and T cells harboring an exhausted phenotype exclusively in the CLL lymph node tissue. Integration of TCR data revealed a clonal expansion of CD8+ precursor exhausted T cells, suggesting their reactivity for CLL cells. Interactome analyses identified the TIM3 ligand Galectin-9 as novel immunoregulatory molecule in CLL. Blocking of Galectin-9 in CLL-bearing mice slowed down disease development and reduced the number of TIM3 expressing T cells. Galectin-9 expression correlated with shorter survival of CLL patients. Thus, Galectin-9 contributes to immune escape in CLL and represents a novel target for immunotherapy.

show abstract

Multi-modal digital pathology for colorectal cancer diagnosis by high-plex immunofluorescence imaging and traditional histology of the same tissue section

Cited by 7 publications

References 95 publications

MIM-CyCIF: Masked Imaging Modeling for Enhancing Cyclic Immunofluorescence (CyCIF) with Panel Reduction and Imputation

MIM-CyCIF: Masked Imaging Modeling for Enhancing Cyclic Immunofluorescence (CyCIF) with Panel Reduction and Imputation

3D multiplexed tissue imaging reconstruction and optimized region-of-interest (ROI) selection through deep learning model of channels embedding

High-dimensional single-cell definition of CLL T cells identifies Galectin-9 as novel immunotherapy target

Contact Info

Product

Resources

About