A comparison of per sample global scaling and per gene normalization methods for differential expression analysis of RNA-seq data

Abstract

Normalization is an essential step with considerable impact on high-throughput RNA sequencing (RNA-seq) data analysis. Although there are numerous methods for read count normalization, it remains a challenge to choose an optimal method due to multiple factors contributing to read count variability that affects the overall sensitivity and specificity. In order to properly determine the most appropriate normalization methods, it is critical to compare the performance and shortcomings of a representative set of normalization routines based on different dataset characteristics. Therefore, we set out to evaluate the performance of the commonly used methods (DESeq, TMM-edgeR, FPKM-CuffDiff, TC, Med UQ and FQ) and two new methods we propose: Med-pgQ2 and UQ-pgQ2 (per-gene normalization after per-sample median or upper-quartile global scaling). Our per-gene normalization approach allows for comparisons between conditions based on similar count levels. Using the benchmark Microarray Quality Control Project (MAQC) and simulated datasets, we performed differential gene expression analysis to evaluate these methods. When evaluating MAQC2 with two replicates, we observed that Med-pgQ2 and UQ-pgQ2 achieved a slightly higher area under the Receiver Operating Characteristic Curve (AUC), a specificity rate > 85%, the detection power > 92% and an actual false discovery rate (FDR) under 0.06 given the nominal FDR (≤0.05). Although the top commonly used methods (DESeq and TMM-edgeR) yield a higher power (>93%) for MAQC2 data, they trade off with a reduced specificity (<70%) and a slightly higher actual FDR than our proposed methods. In addition, the results from an analysis based on the qualitative characteristics of sample distribution for MAQC2 and human breast cancer datasets show that only our gene-wise normalization methods corrected data skewed towards lower read counts. However, when we evaluated MAQC3 with less variation in five replicates, all methods performed similarly. Thus, our proposed Med-pgQ2 and UQ-pgQ2 methods perform slightly better for differential gene analysis of RNA-seq data skewed towards lowly expressed read counts with high variation by improving specificity while maintaining a good detection power with a control of the nominal FDR level.

Evaluation of RNA-seq normalization methods using challenging datasets

“…Previous assessments of RNA-seq normalization methods have usually found that common, widely used methods perform well [2][3][4][5][6]. I show here that this is not the case when challenging normalization tasks are considered, suggesting that previous tests may have been overly simplistic.…”

Sample size calculations and normalization methods for RNA-seq data.

“…Since this study has been recently published, the normalized MAQC2 data from Med-pgQ2, UQ-pgQ2, DESeq and TMM-edgeR and DEGs analysis from these methods can be downloaded in Supporting Information S2-S5 Datasets [81]. Moreover, these normalization methods are written in R (v3.1.3) with the source codes publically available in S1 File (.R) [81]. …”

“…DESeq and TMM normalization methods were implemented using DESeq2 and edgeR packages. UQ-pgQ2 normalization was implemented using R [17,18,81].…”

“…A comparison of these methods with DESeq from DESeq2 and TMM-edgeR normalizations using the benchmark Microarray Quality Control Project datasets (MAQC2) reported that Med-pgQ2 or UQ-pgQ2 perform slightly better for genes with high read counts via improving the specificity for skewed RNA-seq data given a FDR of 0.05. However, these methods also show a slightly higher FP rate for genes with a low read counts compared to DESeq2 and edgeR [81].…”

“…were proposed by Li et al [81]. A comparison of these methods with DESeq from DESeq2 and TMM-edgeR normalizations using the benchmark Microarray Quality Control Project datasets (MAQC2) reported that Med-pgQ2 or UQ-pgQ2 perform slightly better for genes with high read counts via improving the specificity for skewed RNA-seq data given a FDR of 0.05.…”