Estimating False Discovery Proportion Under Arbitrary Covariance Dependence

Abstract

Multiple hypothesis testing is a fundamental problem in high dimensional inference, with wide applications in many scientific fields. In genome-wide association studies, tens of thousands of tests are performed simultaneously to find if any SNPs are associated with some traits and those tests are correlated. When test statistics are correlated, false discovery control becomes very challenging under arbitrary dependence. In the current paper, we propose a novel method based on principal factor approximation, which successfully subtracts the common dependence and weakens significantly the correlation structure, to deal with an arbitrary dependence structure. We derive an approximate expression for false discovery proportion (FDP) in large scale multiple testing when a common threshold is used and provide a consistent estimate of realized FDP. This result has important applications in controlling FDR and FDP. Our estimate of realized FDP compares favorably with Efron (2007)’s approach, as demonstrated in the simulated examples. Our approach is further illustrated by some real data applications. We also propose a dependence-adjusted procedure, which is more powerful than the fixed threshold procedure.

Estimation of the Continuous and Discontinuous Leverage Effects

“…Similar to the case of integrated volatility, the estimation here will not be a statistical problem in the usual sense, since the objects to be estimated are random variables, instead of parameters. Similar examples in statistics include estimating residuals in regression, random effects in mixed effects models, and false discovery proportion (Fan et al, 2012). …”

Projected Principal Component Analysis in Factor Models

“…For instance, y t = (y 1t , ···, y pt )′ can be expression profiles or blood oxygenation level dependent (BOLD) measurements for the t th microarray, proteomic or fMRI-image, whereas i represents a gene or protein or a voxel. See, for example, Desai and Storey (2012); Efron (2010); Fan et al (2012); Friguet et al (2009); Leek and Storey (2008). The separations between the common factors and idiosyncratic components are carried out by the low-rank plus sparsity decomposition.…”

svaseq: removing batch effects and other unwanted noise from sequencing data

“…For step one in the sva algorithm, identifying probes only associated with batch, it has been proposed to use control probes [5,10]. For step two of the sva algorithm, estimating latent factors only associated with dependence, it has been proposed to use factor analysis [12], independent components analysis [13], and principal components analysis [14]. Another extension of the surrogate variable approach in step two has been to model known sources of technical or biological covariation between the measurements for probes, for example in eQTL studies [15,16].…”