Self-Organizing Maps for Cellular In Silico Staining and Cell Substate Classification

Abstract

Cellular composition and structural organization of cells in the tissue determine effective antitumor response and can predict patient outcome and therapy response. Here we present Seg-SOM, a method for dimensionality reduction of cell morphology in H&E-stained tissue images. Seg-SOM resolves cellular tissue heterogeneity and reveals complex tissue architecture. We leverage a self-organizing map (SOM) artificial neural network to group cells based on morphological features like shape and size. Seg-SOM allows for cell segmentation, systematic classification, and in silico cell labeling. We apply the Seg-SOM to a dataset of breast cancer progression images and find that clustering of SOM classes reveals groups of cells corresponding to fibroblasts, epithelial cells, and lymphocytes. We show that labeling the Lymphocyte SOM class on the breast tissue images accurately estimates lymphocytic infiltration. We further demonstrate how to use Seq-SOM in combination with non-negative matrix factorization to statistically describe the interaction of cell subtypes and use the interaction information as highly interpretable features for a histological classifier. Our work provides a framework for use of SOM in human pathology to resolve cellular composition of complex human tissues. We provide a python implementation and an easy-to-use docker deployment, enabling researchers to effortlessly featurize digitalized H&E-stained tissue.

Neural-net-based cell deconvolution from DNA methylation reveals tumor microenvironment associated with cancer prognosis

“…Additionally, to determine the consistency between MEnet and the standard histopathological method for FFPE sections, we compared the lymphocyte frequencies estimated by MEnet with those determined from HE-stained section images. For the HE-stained sections, we employed Seg-SOM ( 66 ), a computational vision map based on an artificial neural network, to identify and classify cell types. Seg-SOM was trained on a comprehensive dataset of expertly annotated cell types and has been validated for accurate cell type identification in various tissues.…”

Characterization and digital spatial deconvolution of the immune microenvironment of intraductal oncocytic papillary neoplasms (IOPN) of the pancreas

“…Then, neighborhoods were clustered into regions based on their cellular composition using “Self-Organizing Map” (SOM) model [ 28 ], with the number of regions determined by the Davies-Bouldin criterion [ 29 ]. Cellular composition was defined as the number of cells specifically positive for a given IHC marker in each neighborhood, divided by the total number of cells in that neighborhood.…”

Expanding the coverage of spatial proteomics

“…One approach for addressing this problem is 'in silico labeling', in which deep learning models are used to predict unmeasured signals from easily acquired reference signals. Examples include predicting subcellular components from unlabeled microscope images (2)(3)(4)(5), virtual histological staining of tissue images (6)(7)(8), and predicting immunofluorescence or directly inferring cell types from immunohistochemically stained images (9,10). This concept can be extended to predict a large number of biomarkers from images of a smaller number (11,12).…”