ZAP:Z-Value Adaptive Procedures for False Discovery Rate Control with Side Information

Abstract

Adaptive multiple testing with covariates is an important research direction that has gained major attention in recent years. It has been widely recognised that leveraging side information provided by auxiliary covariates can improve the power of false discovery rate (FDR) procedures. Currently, most such procedures are devised with p‐values as their main statistics. However, for two‐sided hypotheses, the usual data processing step that transforms the primary statistics, known as z‐values, into p‐values not only leads to a loss of information carried by the main statistics, but can also undermine the ability of the covariates to assist with the FDR inference. We develop a z‐value based covariate‐adaptive (ZAP) methodology that operates on the intact structural information encoded jointly by the z‐values and covariates. It seeks to emulate the oracle z‐value procedure via a working model, and its rejection regions significantly depart from those of the p‐value adaptive testing approaches. The key strength of ZAP is that the FDR control is guaranteed with minimal assumptions, even when the working model is misspecified. We demonstrate the state‐of‐the‐art performance of ZAP using both simulated and real data, which shows that the efficiency gain can be substantial in comparison with p‐value‐based methods. Our methodology is implemented in the R package zap.

How to train a post-processor for tandem mass spectrometry proteomics database search while maintaining control of the false discovery rate

“…This paper focuses on the peptide detection problem; however, the two RESET procedures are more generally applicable to competition-based FDR control with side information. The latter problem, as well as the related problem of using side-information with p-value based control of the FDR, have been also studied recently in the statistics community with tools such as ZAP [18] AdaPT [17], Adaptive Knockoffs [24] offering finite sample FDR control. However, none of these tools are applicable in our context: ZAP assumes a parametric model where p-values can be computed, AdaPT addresses both the p-value and the competition-based contexts but only provides a meta scheme in the latter case, and, as we argue below, Adaptive Knockoffs is forbiddingly slow for a typical peptide detection problem, and moreover, when run to completion on a small dataset it offers significantly fewer discoveries than Percolator-RESET does.…”