DECONbench: a benchmarking platform dedicated to deconvolution methods for tumor heterogeneity quantification

Decamps, Clémentine; Arnaud, A; Petitprez, Florent; Ayadi, Mira; Baurès, Aurélia; Armenoult, Lucile; Nicolle, Rémy; Tomasini, Richard; Reyniès, Aurélien de; Cros, Jérôme; Blum, Yuna; Richard, Magali

doi:10.1101/2020.06.06.131482

Cited by 3 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With respect to the first of these, DNA methylation profiles have also proven to be a valuable source of information for tumor content deconvolution and it is possible to integrate multiple modalities to improve accuracy. 38 Nevertheless, the generated data here offer two principal advantages: they allow the objective definition of ground truth used in our evaluation and they can now be flexibly applied by others. In particular, the purified expression profiles can be used to simulate other admixtures in silico in proportions relevant to particular and diverse disease settings.…”

Section: Discussionmentioning

confidence: 99%

Community assessment of methods to deconvolve cellular composition from bulk gene expression

White

Reyniès

Newman

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deconvolution methods infer levels of immune and stromal infiltration from bulk expression of tumor samples. These methods allow projection of characteristics of the tumor microenvironment, known to affect patient outcome and therapeutic response, onto the millions of bulk transcriptional profiles in public databases, many focused on uniquely valuable and clinically-annotated cohorts. Despite the wide development of such methods, a standardized dataset with ground truth to evaluate their performance has been lacking. We generated and sequenced in vitro and in silico admixtures of tumor, immune, and stromal cells and used them as ground truth in a community-wide DREAM Challenge that provided an objective, unbiased assessment of six widely-used published deconvolution methods and of 22 new analytical approaches developed by international teams. Our results demonstrate that existing methods predict many cell types well, while team-contributed methods highlight the potential to resolve functional states of T cells that were either not covered by published reference signatures or estimated poorly by some published methods. Our assessment and the open-source implementations of top-performing methods will allow researchers to apply the deconvolution approach most appropriate to querying their cell type of interest. Further, our publicly-available admixed and purified expression profiles will be a valuable resource to those developing deconvolution methods, including in non-malignant settings involving immune cells.

show abstract

Section: Discussionmentioning

confidence: 99%

Community assessment of methods to deconvolve cellular composition from bulk gene expression

White

Reyniès

Newman

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Documentation. The documentation is organized according to stakeholders categories organizers, administrators, and contributors directly on the first page of the documentation 4 . As an organizer, you are accompanied with several benchmark templates, from simple to elaborate, to ease the technical aspects of building a benchmark, and to let you concentrate on scientific aspects of the benchmark.…”

Section: Other Featuresmentioning

confidence: 99%

“…In this section, we introduce DECONbench [4] for benchmarking deconvolution methods inferring the tumor micro-environment composition. We show two features of Codabench: (1) flexibility of benchmark bundle (in this use case, another task and programming language R supported) (2) reusability and portability of benchmark bundles.…”

Section: Use Case 2: Deconbench Benchmarkmentioning

confidence: 99%

Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform

Xu,

Escalera,

Guyon

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Obtaining standardized crowdsourced benchmark of computational methods is a major issue in scientific communities. Dedicated frameworks enabling fair continuous benchmarking in a unified environment are yet to be developed. Here we introduce Codabench, an open-sourced, community-driven platform for benchmarking algorithms or software agents versus datasets or tasks. A public instance of Codabench is open to everyone, free of charge, and allows benchmark organizers to compare fairly submissions, under the same setting (software, hardware, data, algorithms), with custom protocols and data formats. Codabench has unique features facilitating the organization of benchmarks flexibly, easily and reproducibly. Firstly, it supports code submission and data submission for testing on dedicated compute workers, which can be supplied by the benchmark organizers. This makes the system scalable, at low cost for the platform providers. Secondly, Codabench benchmarks are created from self-contained "bundles", which are zip files containing a full description of the benchmark in a configuration file (following a well-defined schema), documentation pages, data, ingestion and scoring programs, making benchmarks reusable and portable. The Codabench documentation includes many examples of bundles that can serve as templates. Thirdly, Codabench uses dockers for each task's running environment to make results reproducible. Codabench has been used internally and externally with more than 10 applications during the past 6 months. As illustrative use cases, we introduce 4 diverse benchmarks covering Graph Machine Learning, Cancer Heterogeneity, Clinical Diagnosis and Reinforcement Learning.Preprint. Under review.

show abstract

“…However, there is still no available 'gold standard' dataset to evaluate the ability of an algorithm using these proteomics-defined immune cell types to deconvolve tumor data. On the other side, for mRNA-based deconvolution, there are numerous single-cell datasets as well as sorted cell experiments that can be used for such benchmarks [31][32][33] which are missing for proteomic data.…”

Section: Introductionmentioning

confidence: 99%

Decomprolute: A benchmarking platform designed for multiomics-based tumor deconvolution

Feng

Calinawan

Pugliese

et al. 2023

Preprint

View full text Add to dashboard Cite

Tumor deconvolution is a reliable way to disentangle the diverse cell types that comprise solid tumors. To date, however, both the algorithms developed to deconvolve tumor samples, and the gold standard datasets used to assess the algorithms are geared toward the analysis of gene expression (e.g., RNA-seq) rather than protein levels in tumor cells. While gene expression is less expensive to measure, protein levels provide a more accurate view of immune markers. To facilitate the development as well as improve the reproducibility and reusability of multi-omic deconvolution algorithms, we introduce Decomprolute, a Common Workflow Language framework that leverages containerization to compare tumor deconvolution algorithms across multiomic data sets. Decomprolute incorporates the large-scale multiomic data sets produced by the Clinical Proteomic Tumor Analysis Consortium (CPTAC), which include matched mRNA expression and proteomic data from thousands of tumors across multiple cancer types to build a fully open-source, containerized proteogenomic tumor deconvolution benchmarking platform. The platform consists of modular architecture and it comes with well-defined input and output formats at each module. As a result, it is robust and extendable easily with additional algorithms or analyses. The platform is available for access and uses at http://pnnl-compbio.github.io/decomprolute.

show abstract

DECONbench: a benchmarking platform dedicated to deconvolution methods for tumor heterogeneity quantification

Cited by 3 publications

References 11 publications

Community assessment of methods to deconvolve cellular composition from bulk gene expression

Community assessment of methods to deconvolve cellular composition from bulk gene expression

Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform

Decomprolute: A benchmarking platform designed for multiomics-based tumor deconvolution

Contact Info

Product

Resources

About