Benchmarking Procedures for High-Throughput Context Specific Reconstruction Algorithms

Abstract

Recent progress in high-throughput data acquisition has shifted the focus from data generation to processing and understanding of how to integrate collected information. Context specific reconstruction based on generic genome scale models like ReconX or HMR has the potential to become a diagnostic and treatment tool tailored to the analysis of specific individuals. The respective computational algorithms require a high level of predictive power, robustness and sensitivity. Although multiple context specific reconstruction algorithms were published in the last 10 years, only a fraction of them is suitable for model building based on human high-throughput data. Beside other reasons, this might be due to problems arising from the limitation to only one metabolic target function or arbitrary thresholding. This review describes and analyses common validation methods used for testing model building algorithms. Two major methods can be distinguished: consistency testing and comparison based testing. The first is concerned with robustness against noise, e.g., missing data due to the impossibility to distinguish between the signal and the background of non-specific binding of probes in a microarray experiment, and whether distinct sets of input expressed genes corresponding to i.e., different tissues yield distinct models. The latter covers methods comparing sets of functionalities, comparison with existing networks or additional databases. We test those methods on several available algorithms and deduce properties of these algorithms that can be compared with future developments. The set of tests performed, can therefore serve as a benchmarking procedure for future algorithms.

Increasing consensus of context-specific metabolic models by integrating data-inferred cell functions

“…Model reaction content is often evaluated to compare context-specific algorithms. Recently,130 approaches to benchmark models with their functionalities have been proposed (Opdam et al, 131 2017;Agren et al, 2014;Pacheco et al, 2016). Current approaches use repositories of known 132 cellular tasks to assess the capacity of models to achieve specific modeling goals or to enable 133 the representation of specific metabolic functions.…”

A Systematic Evaluation of Methods for Tailoring Genome-Scale Metabolic Models

“…Initial procedures for testing context-specific models have been proposed (Pacheco et al, 2015), but the validity and implications of the assumptions underlying the extraction method have not been rigorously tested for mammalian models. Thus, here we evaluate the performance of several prominent model extraction algorithms, analyze the impact of their assumptions, and quantify the influence of key decisions that must be made when building cell line- and tissue-specific models.…”

Modeling metabolic variation with single-cell expression data

“…We construct context specific metabolic models based on the reaction confidence scores using the Cost Optimization Reaction Dependency Assessment (CORDA) algorithm (Schultz and Qutub, 2016) . We choose CORDA among many other construction algorithms (Pacheco et al, 2015) due to its efficiency and discretization of reaction confidence scores. CORDA uses linear programming to construct context specific metabolic models, so its time complexity is tractable in constructing thousands of models.…”

“…Analysis of single-cell RNA-seq data allows for the definition of "cell type" based on shared characteristics in single-cell transcriptomic profiles. This type of high-resolution transcriptomic characterization of cells and tissues offers significant potential in understanding the function of cells and tissues, including cell-and tissue type specific metabolic processes that can be inferred through context specific metabolic modeling (Pacheco et al, 2015) . Metabolism can be modeled as a collection of metabolite conversions within a cellular compartment and exchanges between different subcellular compartments, such as the cytoplasm, nucleus, and mitochondrial matrix ( Figure S1A and S1B ).…”