A Non-Randomized Procedure for Large-Scale Heterogeneous Multiple Discrete Testing Based on Randomized Tests

Abstract

In the analysis of next‐generation sequencing technology, massive discrete data are generated from short read counts with varying biological coverage. Conducting conditional hypothesis testing such as Fisher's Exact Test at every genomic region of interest thus leads to a heterogeneous multiple discrete testing problem. However, most existing multiple testing procedures for controlling the false discovery rate (FDR) assume that test statistics are continuous and become conservative for discrete tests. To overcome the conservativeness, in this article, we propose a novel multiple testing procedure for better FDR control on heterogeneous discrete tests. Our procedure makes decisions based on the marginal critical function (MCF) of randomized tests, which enables achieving a powerful and non‐randomized multiple testing procedure. We provide upper bounds of the positive FDR (pFDR) and the positive false non‐discovery rate (pFNR) corresponding to our procedure. We also prove that the set of detections made by our method contains every detection made by a naive application of the widely‐used q‐value method. We further demonstrate the improvement of our method over other existing multiple testing procedures by simulations and a real example of differentially methylated region (DMR) detection using whole‐genome bisulfite sequencing (WGBS) data.

Multiple testing of composite null hypotheses for discrete data using randomized p‐values

“…The authors showed that a simple randomization technique, which is not only limited to sequencing data but can be used for general discrete test statistics, can address this issue. In the case of multiple testing of heterogeneous discrete data, Dai et al (2019) proposed a procedure based on the marginal critical function (MCF) of randomized tests. The MCF approach provides a nonrandom decision that is based on ranked MCF values, although it utilizes randomized tests.…”

“…In the case of multiple testing of heterogeneous discrete data, Dai et al. (2019) proposed a procedure based on the marginal critical function (MCF) of randomized tests. The MCF approach provides a nonrandom decision that is based on ranked MCF values, although it utilizes randomized tests.…”