Derandomized knockoffs: leveraging e-values for false discovery rate control

Abstract

Model-X knockoffs is a flexible wrapper method for high-dimensional regression algorithms, which provides guaranteed control of the false discovery rate (FDR). Due to the randomness inherent to the method, different runs of model-X knockoffs on the same dataset often result in different sets of selected variables, which is undesirable in practice. In this paper, we introduce a methodology for derandomizing model-X knockoffs with provable FDR control. The key insight of our proposed method lies in the discovery that the knockoffs procedure is in essence an e-BH procedure. We make use of this connection, and derandomize model-X knockoffs by aggregating the e-values resulting from multiple knockoff realizations. We prove that the derandomized procedure controls the FDR at the desired level, without any additional conditions (in contrast, previously proposed methods for derandomization are not able to guarantee FDR control). The proposed method is evaluated with numerical experiments, where we find that the derandomized procedure achieves comparable power and dramatically decreased selection variability when compared with model-X knockoffs.

Notip: Non-parametric true discovery proportion control for brain imaging

“…However, the two existing aggregation schemes (AKO and e-values) that formally guarantee FDR control are generally conservative and achieve FDP control empirically. This is consistent with the findings of [18]. The Closed Testing procedure of [13] achieves FDP control as announced but suffers from a lack of power.…”

“…Additionally, we set k max = ⌊p/50⌋ following the approach of [3]. We also consider both state-of-the-art Knockoffs aggregation schemes: AKO (Aggregation of Multiple Knockoffs, 17) and e-values based aggregation [18]. Additionally, we consider Vanilla Knockoffs, i.e.…”

“…To mitigate this, aggregation of multiple Knockoffs runs is needed. Ren and Barber [18] introduced an aggregation scheme which relies defining Knockoffs e-values.…”

“…Much work has been done recently to tackle this issue, including using stability selection criteria (Ren, Wei and Candès, 2021), aggregating multiple π-statistics (Nguyen et al, 2020;Blain et al, 2023) and using aggregated Knockoffs e-values (Ren and Barber, 2022). The gist of these approaches is to draw multiple sets of Knockoffs and aggregate the resulting statistics or rejection sets in order to robustify the variable selection process.…”

Controlling for false discoveries subsequently to large scale one‐way ANOVA testing in proteomics: Practical considerations

“…Unfortunately, the theory underlying FDR control is not trivial and its correct application to biomarker selection remains at the center of an ever-renewed body of literature [2][3][4][5][6][7][8].…”