A multi-encoder variational autoencoder controls multiple transformational features in single-cell image analysis

Abstract

Image-based cell phenotyping relies on quantitative measurements as encoded representations of cells; however, defining suitable representations that capture complex imaging features is challenged by the lack of robust methods to segment cells, identify subcellular compartments, and extract relevant features. Variational autoencoder (VAE) approaches produce encouraging results by mapping an image to a representative descriptor, and outperform classical hand-crafted features for morphology, intensity, and texture at differentiating data. Although VAEs show promising results for capturing morphological and organizational features in tissue, single cell image analyses based on VAEs often fail to identify biologically informative features due to uninformative technical variation. Here we propose a multi-encoder VAE (ME-VAE) in single cell image analysis using transformed images as a self-supervised signal to extract transform-invariant biologically meaningful features, including emergent features not obvious from prior knowledge. We show that the proposed architecture improves analysis by making distinct cell populations more separable compared to traditional and recent extensions of VAE architectures and intensity measurements by enhancing phenotypic differences between cells and by improving correlations to other analytic modalities. Better feature extraction and image analysis methods enabled by the ME-VAE will advance our understanding of complex cell biology and enable discoveries previously hidden behind image complexity ultimately improving medical outcomes and drug discovery.

Computational Multiplex Panel Reduction to Maximize Information Retention in Breast Cancer Tissue Microarrays

“…For selecting an optimal reduced panel set, we evaluate 4 different methods: correlation-based selection, sparse subspace-based selection, gradient-based selection, and random selection as shown in Figure 1 (see Methods). The reduced panels of each method were then used to reconstruct the initial full panel using a multi-encoder variational autoencoder (ME-VAE) 9 , which encodes the markers of single cell image in the reduced panel into a latent descriptor and generates all 25 markers of single cell image in the full panel set. The reconstructed images of each method are then evaluated using various metrics including single-cell based structural similarity index measure (SSIM) from the reconstructed image, mean intensity correlation between real stained and the reconstructed image, and cluster overlap to determine whether information is retained in the reduced panel and prediction pipeline.…”

“…Although it is important to be able to reconstruct the mean intensities of single cells, downstream analysis such as single cell phenotyping and clustering is important for biological research, and if such analytical methods were to be affected, then the reduced panel predictions would not be useful for complex research methods. As shown in Figure 6, although the selection methods have varied levels of performance at predicting mean intensity, when 18 of 25 markers are included in the reduced panel sets, all selection methods perform well at recapturing the same clusters extracted from the full panel set, as measured by normalized mutual information (NMI) 9 : where U and V are the reduced panel predicted and full panel (ground truth) cluster labels and H(U) and H(V) represent the entropy of U and V, respectively. The predicted clusters were then paired to their full panel counterpart by examining the population compositions to maximize consistency.…”

“…For selecting an optimal reduced panel set, we evaluate 4 different methods: correlation-based selection, sparse subspace-based selection, gradient-based selection, and random selection as shown inFigure1(see Methods). The reduced panels of each method were then used to reconstruct the initial full panel using a multi-encoder variational autoencoder (ME-VAE)9 , which encodes the markers of single cell image in the reduced panel into a latent descriptor and generates all 25 markers of single cell image in the full panel set.…”

Deciphering the Immune Complexity in Esophageal Adenocarcinoma and Pre-Cancerous Lesions With Sequential Multiplex Immunohistochemistry and Sparse Subspace Clustering Approach

“…Third, all multiplex imaging methods quantify marker expression through the signal deconvolution of corresponding images, which is digitally achieved, involving three major steps: nuclear segmentation that recognizes the cell nucleus and marks the boundary of the cell; deconvolution of raw images to isolate signal intensities that are measured within the boundary of the cell; and the conversion of the signal intensity into a single value that represents the marker expression level. In general, methods with higher image resolution will lead to more accurate quantification and allow for more sophisticated computation algorithms ( 55 ). Microscopic-based imaging methods, such as mIHC, CyCIF ( 53 ), and CODEX ( 54 ) yield higher-resolution images compared with mass spectroscopy-based Image CyTOF ( 51 ).…”

Deformability Cytometry Clustering with Variational Autoencoders

“…VAEs reduces the dimensionality of input data to arbitrary dimensions 16 and has been previously used for clustering cell data. 8,9…”