Evaluating single-subject study methods for personal transcriptomic interpretations to advance precision medicine

Abstract

Background: Gene expression profiling has benefited medicine by providing clinically relevant insights at the molecular candidate and systems levels. However, to adopt a more 'precision' approach that integrates individual variability including 'omics data into risk assessments, diagnoses, and therapeutic decision making, whole transcriptome expression analysis requires methodological advancements. One need is for users to confidently be able to make individual-level inferences from whole transcriptome data. We propose that biological replicates in isogenic conditions can provide a framework for testing differentially expressed genes (DEGs) in a single subject (ss) in absence of an appropriate external reference standard or replicates. Methods: Eight ss methods for identifying genes with differential expression (NOISeq, DEGseq, edgeR, mixture model, DESeq, DESeq2, iDEG, and ensemble) were compared in Yeast (parental line versus snf2 deletion mutant; n=42/condition) and MCF7 breast-cancer cell (baseline and stimulated with estradiol; n=7/condition) RNA-Seq datasets where replicate analysis was used to build reference standards from NOISeq, DEGseq, edgeR, DESeq, DESeq2. Each dataset was randomly partitioned so that approximately two-thirds of the paired samples were used to construct reference standards and the remainder were treated separately as single-subject sample pairs and DEGs were assayed using ss methods. Receiver-operator characteristic (ROC) and precision-recall plots were determined for all ss methods against each RSs in both datasets (525 combinations). Results: Consistent with prior analyses of these data, ~50% and ~15% DEGs were respectively obtained in Yeast and MCF7 reference standard datasets regardless of the analytical method. NOISeq, edgeR and DESeq were the most concordant and robust methods for creating a reference standard. Single-subject versions of NOISeq, DEGseq, and an ensemble learner achieved the best median ROC-area-under-thecurve to compare two transcriptomes without replicates regardless of the type of reference standard (>90% in Yeast, >0.75 in MCF7). Conclusion: Better and more consistent accuracies are obtained by an ensemble method applied to singlesubject studies across different conditions. In addition, distinct specific sing-subject methods perform better according to different proportions of DEGs. Single-subject methods for identifying DEGs from paired samples need improvement, as no method performs with both precision>90% and recall>90%. http://www.lussiergroup.org/publications/EnsembleBiomarker

Autologous micrograft accelerates endogenous wound healing response through ERK-induced cell migration

“…Using the N-of-1 Pathway MixEnrich analysis [16], we identified DEGs without the requirement of large cohorts or replicates by directly analyzing paired samples (AMGtreated vs untreated cells) drawn from the same animal upon different AMG treatment time points (1 h, 5 h, 12 h, and 24 h). All samples have been first normalized by using NOIseq [17,18]. Next, for each transcriptome sample we computed the absolute value of log-transformed fold change |log 2 FC| as |log 2 (U/T)|, where U and T are the gene expression level in the untreated and AMG-treated condition, respectively.…”

Personalized beyond Precision: Designing Unbiased Gold Standards to Improve Single-Subject Studies of Personal Genome Dynamics from Gene Products

“…Reusing part of the reference standard data for generating predictions creates dependencies, an evaluation framework problem observed more frequently in statistical evaluation of isogenic data. [11,12] This manuscript focuses on improving the accuracy of single-subject studies evaluations, beyond "naïve reproducibility" of results and other biases described in Table 1. For example, in a given dataset, various tools may yield drastically different but still plausible algorithmic solutions depending on data distribution assumptions, resulting in technical noise that muddle the actual biological signal.…”

“…For example, in a given dataset, various tools may yield drastically different but still plausible algorithmic solutions depending on data distribution assumptions, resulting in technical noise that muddle the actual biological signal. In a prior study of 5 distinct RNA analysis methods in multiple isogenic datasets [12], we described a new method that combines the inconsistent signal between analytical methods that was previously unaddressed in the original study [13]. This inconsistency required methods such as DESeq [14]to impose a false discovery rate (FDR) cutoff of 0.001 to detect ~3,000 DEGs, while DEGseq [15] required a cutoff of FDR<3.6x10 -12 for the same number of DEGs, with 2039 overlapping transcripts.…”

“…We have previously created a reference standard from multiple independent methods -without the method evaluatedand shown more conservative accuracy estimates. Specifically, we constructed an ensemble learner [12] to develop reference standards, where the ensemble approach resolves conflicting biomarker prediction, uses no statistical assumptions, and removes anti-conservative isomorphic evaluations. Of note, we avoid using the term gold standard which generally pertains to well-validated biological results and prefer the term reference standard for results constructed through high throughput analytical validations.…”

“…. Our prior study demonstrated that in situations comprising high technical noise, an ensemble learner maximizes the stability of a reference standard and the DEG predictions [12]. However, ensemble learners increase the "black-box" aspect of the data analysis and muddle its interpretability.…”