Evaluating statistical analysis models for RNA sequencing experiments

Abstract

Validating statistical analysis methods for RNA sequencing (RNA-seq) experiments is a complex task. Researchers often find themselves having to decide between competing models or assessing the reliability of results obtained with a designated analysis program. Computer simulation has been the most frequently used procedure to verify the adequacy of a model. However, datasets generated by simulations depend on the parameterization and the assumptions of the selected model. Moreover, such datasets may constitute a partial representation of reality as the complexity or RNA-seq data is hard to mimic. We present the use of plasmode datasets to complement the evaluation of statistical models for RNA-seq data. A plasmode is a dataset obtained from experimental data but for which come truth is known. Using a set of simulated scenarios of technical and biological replicates, and public available datasets, we illustrate how to design algorithms to construct plasmodes under different experimental conditions. We contrast results from two types of methods for RNA-seq: (1) models based on negative binomial distribution (edgeR and DESeq), and (2) Gaussian models applied after transformation of data (MAANOVA). Results emphasize the fact that deciding what method to use may be experiment-specific due to the unknown distributions of expression levels. Plasmodes may contribute to choose which method to apply by using a similar pre-existing dataset. The promising results obtained from this approach, emphasize the need of promoting and improving systematic data sharing across the research community to facilitate plasmode building. Although we illustrate the use of plasmode for comparing differential expression analysis models, the flexibility of plasmode construction allows comparing upstream analysis, as normalization procedures or alignment pipelines, as well.

Excess False Positive Rates in Methods for Differential Gene Expression Analysis using RNA-Seq Data

“…Robles et al (2012) likewise compared 3 methods and found inflated type I error rates for some methods and conservative performance from others. Reeb and Steibel (2013) compared 3 methods using "plasmodes" (resampled data) and found inflated type I error rates for small significance levels. Guo, Li, Ye, and Shyr (2013) compare 6 methods and conclude that all "suffer from over-sensitivity".…”

Homoeologue expression insights into the basis of growth heterosis at the intersection of ploidy and hybridity in Cyprinidae

“…Principal component analysis (PCA) of the twelve liver transcriptomes was applied to examine the contribution of each transcript to the separation of the classes3435. Then, fastq formatted reads from the two diploid parents and two hybrid offspring were mapped to the reference genome using TopHat23637.…”

Determination of dosage compensation and comparison of gene expression in a triploid hybrid fish

“…Transcriptome de novo assembly was carried out with a short-reads assembly program (Trinity) [54], using three independent software modules called Inchworm, Chrysalis, and Butterfly. Principal component analysis (PCA) of nine liver transcriptomes was applied to examine the contribution of each transcript to the separation of the classes [55, 56] (Additional file 9). …”