A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium

Abstract

We present primary results from the Sequencing Quality Control (SEQC) project, coordinated by the United States Food and Drug Administration. Examining Illumina HiSeq, Life Technologies SOLiD and Roche 454 platforms at multiple laboratory sites using reference RNA samples with built-in controls, we assess RNA sequencing (RNA-seq) performance for junction discovery and differential expression profiling and compare it to microarray and quantitative PCR (qPCR) data using complementary metrics. At all sequencing depths, we discover unannotated exon-exon junctions, with >80% validated by qPCR. We find that measurements of relative expression are accurate and reproducible across sites and platforms if specific filters are used. In contrast, RNA-seq and microarrays do not provide accurate absolute measurements, and gene-specific biases are observed, for these and qPCR. Measurement performance depends on the platform and data analysis pipeline, and variation is large for transcript-level profiling. The complete SEQC data sets, comprising >100 billion reads (10Tb), provide unique resources for evaluating RNA-seq analyses for clinical and regulatory settings.

Seasonal phenotype‐specific transcriptional reprogramming during metamorphosis in the European map butterfly Araschnia levana

“…However, the sensitivity and accuracy of RNA‐Seq largely depends on the millions of reads sequenced per sample, the number of replicates used and the filtering and mapping procedures for data processing. For example, in a study comparing RNA‐Seq sequencing depth and the identification of differentially expressed genes, 10 million mapped fragments were sufficient to confirm the differential expression of the most strongly expressed genes but genes with lower expression levels suffered a high FDR (SEQC/MAQC‐III Consortium 2014). Therefore, although the SSH method can yield false positives, the identification of differentially expressed genes with low expression levels by RNA‐Seq requires a greater sequencing depth, and this is more expensive.…”

variancePartition: Interpreting drivers of variation in complex gene expression studies

“…2d). While technical variation in RNA-seq is known to depend on GC content [8,9], variancePartition gives a clear illustration of how the effect of technical artifacts varies substantially across genes. Moreover, this analysis can be used to identify other correlates underlying technical issues in expression variation.…”

“…What is the relative contribution of experimental stimulus versus regulatory genetics to variation in gene expression [5]? Is technical variability of RNA-seq low enough to study regulatory genetics and disease biology, and what are the major drivers of this technical variability [2,8,9]? A rich understanding of complex datasets requires answering these questions with both a genomewide summary and gene-level resolution.…”

RElative QUantitation Inferred by Evaluating Mixtures (REQUIEM)

“…Quantifying sample-to-sample differences (i.e., fold-changes) for each component often provides sufficient information for generating or testing hypotheses, eliminating the need for experimentally more demanding absolute quantification. Such relative quantitation has thus become the standard approach in many analytical disciplines, especially in the analysis of structurally complex biomolecules in highly multiplexed fashion [1–4]. Although many ingenious quantitation methods have been developed in this context, implementing the more powerful ones (e.g., metabolic labeling) is far from routine, as many of these are difficult and/or expensive to perform [5].…”