Performance of computational algorithms to deconvolve heterogeneous bulk tumor tissue depends on experimental factors

Abstract

Background: Single-cell gene expression profiling provides unique opportunities to understand tumor heterogeneity and the tumor microenvironment. Because of cost and feasibility, profiling bulk tumors remains the primary population-scale analytical strategy. Many algorithms can deconvolve these tumors using single-cell profiles to infer their composition. While experimental choices do not change the true underlying composition of the tumor, they can affect the measurements produced by the assay. Results: We generated a dataset of high-grade serous ovarian tumors with paired expression profiles from using multiple strategies to examine the extent to which experimental factors impact the results of downstream tumor deconvolution methods. We find that pooling samples for single-cell sequencing and subsequent demultiplexing has a minimal effect. We identify dissociation-induced differences that affect cell composition, leading to changes that may compromise the assumptions underlying some deconvolution algorithms. We also observe differences across mRNA enrichment methods that introduce additional discrepancies between the two data types. We also find that experimental factors change cell composition estimates and that the impact differs by method. Conclusions: Previous benchmarks of deconvolution methods have largely ignored experimental factors. We find that methods vary in their robustness to experimental factors. We provide recommendations for methods developers seeking to produce the next generation of deconvolution approaches and for scientists designing experiments using deconvolution to study tumor heterogeneity.

Heterogeneous pseudobulk simulation enables realistic benchmarking of cell-type deconvolution methods

“…We also report that both BayesPrism and marker-based methods are robust to heterogeneity and in particular can outperform reference based approaches. Our results align with a recent study [39] that benchmarked deconvolution methods on real bulk and single cell data finding that BayesPrism strongly outperforms all tested reference based methods when evaluated for consistency across different biochemical and bioinformatic processing pipelines for the same biological sample. While the Hippen et al’s study looks at heterogeneity from a different perspective, the fundamental conclusion that reference based methods are highly sensitive to heterogeneity is notably consistent.…”

Benchmark of cellular deconvolution methods using a multi-assay reference dataset from postmortem human prefrontal cortex

“…Several comprehensive benchmarking efforts have been performed by independent groups [17][18][19][20] which evaluated deconvolution methods across a wide range of tissues, simulation scenarios, and normalization methods [17][18][19][20]. However, the performance rankings of different deconvolution methods have mostly been inconsistent due to several plausible reasons, including 1) the tissue for which they were initially developed did not match the tissue used for evaluation, 2) biases and variability in reference sc/snRNA-seq datasets, 3) variability in the selection of cell type marker genes, 4) differences in cell type heterogeneity in the tissue under study, 5) choice of cell type resolution (fine or broad), 6) factors regarding how the tissue samples were extracted or preserved, 7) differences between the target RNA-seq and reference sc/snRNA-seq data regarding cell fractions profiled and library preparation strategies, and 8) differences in data normalization and processing [17][18][19][20][21][22].…”

“…Some deconvolution methods already model statistical differences between sequencing assays [6], while others employ normalization methods. Ultimately, the RNA extraction method and library preparation protocol used for bulk RNA-seq can impact benchmarking results of deconvolution methods [22].…”

InstaPrism: an R package for fast implementation of BayesPrism

“…Among numerous deconvolution methodologies that have been developed so far, BayesPrism [8] provides a novel Bayesian deconvolution framework that infers cell-type specific proportions and expression states using a reference scRNAseq dataset from similar tissue samples. Extensive analysis in the original BayePrism as well as independent deconvolution benchmarking studies [9,10] have suggested that BayesPrism is superior to previous methods under different realistic settings. Specifically, BayesPrism appears highly robust to different sources of model misspecification that can introduce bias into standard reference-regression based methods, such as biological heterogeneity within cell-types [9] or technical differences between single cell reference and bulk [10])…”

“…Extensive analysis in the original BayePrism as well as independent deconvolution benchmarking studies [9,10] have suggested that BayesPrism is superior to previous methods under different realistic settings. Specifically, BayesPrism appears highly robust to different sources of model misspecification that can introduce bias into standard reference-regression based methods, such as biological heterogeneity within cell-types [9] or technical differences between single cell reference and bulk [10])…”