The winner’s curse under dependence: repairing empirical Bayes using convoluted densities

Abstract

The winner′s curse is a form of selection bias that arises when estimates are obtained for a large number of features, but only a subset of most extreme estimates is reported. It occurs in large scale significance testing as well as in rank-based selection, and imperils reproducibility of findings and follow-up study design. Several methods correcting for this selection bias have been proposed, but questions remain on their relative performance and susceptibility to dependence between features since comparative studies are missing. We demonstrate how empirical Bayes methods tackling selection bias require a custom density estimator for competitive performance under dependence. Furthermore, we perform a comprehensive simulation study comparing different classes of correction methods for point estimates as well as confidence intervals, while studying the effect of dependence. We find a bootstrap method by Tan et al. (2015) and empirical Bayes methods to perform best overall for correcting the selection bias, although this correction does not always improve the feature ranking. We apply selection bias correction methods to a comparison of the best single-feature prediction model versus the multi-feature model in predicting Brassica napus phenotypes from gene expression data, demonstrating that the superiority of the best single-feature model may be illusory.