Should We Abandon the t-Test in the Analysis of Gene Expression Microarray Data: A Comparison of Variance Modeling Strategies

Abstract

High-throughput post-genomic studies are now routinely and promisingly investigated in biological and biomedical research. The main statistical approach to select genes differentially expressed between two groups is to apply a t-test, which is subject of criticism in the literature. Numerous alternatives have been developed based on different and innovative variance modeling strategies. However, a critical issue is that selecting a different test usually leads to a different gene list. In this context and given the current tendency to apply the t-test, identifying the most efficient approach in practice remains crucial. To provide elements to answer, we conduct a comparison of eight tests representative of variance modeling strategies in gene expression data: Welch's t-test, ANOVA [1], Wilcoxon's test, SAM [2], RVM [3], limma [4], VarMixt [5] and SMVar [6]. Our comparison process relies on four steps (gene list analysis, simulations, spike-in data and re-sampling) to formulate comprehensive and robust conclusions about test performance, in terms of statistical power, false-positive rate, execution time and ease of use. Our results raise concerns about the ability of some methods to control the expected number of false positives at a desirable level. Besides, two tests (limma and VarMixt) show significant improvement compared to the t-test, in particular to deal with small sample sizes. In addition limma presents several practical advantages, so we advocate its application to analyze gene expression data.

Nephron Toxicity Profiling via Untargeted Metabolome Analysis Employing a High Performance Liquid Chromatography-Mass Spectrometry-based Experimental and Computational Pipeline

“…Significantly regulated metabolites were evaluated by pairwise comparison of control to low or high concentration for each time point using LIMMA, which has a higher discriminatory power than t tests without increasing the false positive rate, especially when dealing with small sample sizes (39). The returned p values were corrected for multiple testing according to Hochberg and Benjamini (35).…”

Role of the Yap Family in the Transcriptional Response to Oxidative Stress in Yeasts

“…Besides data pre-processing, we used a combination of three different algorithms: SAM (Tusher et al, 2001), LIMMA (Wettenhall and Smyth, 2004) and SMVar (Jaffrezic et al, 2007). These algorithms were chosen because they are representative of different variance modelling strategies in gene expression data (Jeanmougin et al, 2010). In total, 786 genes were identified as being significantly up regulated in S. cerevisiae, 327 genes in C. glabrata and 337 genes in C. albicans ( Figure 1, step 1).…”

An improved analysis methodology for translational profiling by microarray

“…Indeed, these methods assume that (i) only a small fraction of genes are significantly deregulated, and (ii) this deregulation occurs symmetrically around zero (i.e., there is an equal number of up-and down-regulated genes) . While these assumptions may hold true for transcriptional profiling (for which all these methods were initially developed), they can be easily violated with skewed experiments (due to the study design and/or to the nature of the biological samples) such as enrichment experiments (e.g., CHromatin Immuno Precipitation) or sensitive systems where deregulation is global and/or asymmetrical, such as in many studies of the regulation of translation (changes in the "translatome") (Jeanmougin et al 2010;Landfors et al 2011). When these assumptions are violated, general microarray methods will introduce inferential errors leading to spurious deregulation (false positives) and/or censored biological changes (false negatives) (Zhao and Pan 2003;Dabney and Storey 2007).…”