On the Optimality of Some Multiple Comparison Procedures

Spjøtvoll, Emil

doi:10.1214/aoms/1177692621

Cited by 85 publications

(68 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The two procedures are different when p k varies across groups. As argued by Storey (2007), the setting considered in Spjøtvoll (1972) is problematic because of the wrong definitions of "ETP" and "EFP," which do not represent the underlying reality. It can be shown that this wrong formulation naturally leads to a procedure that ignores p k , which provides important information for ranking the hypotheses.…”

Section: Discussionmentioning

confidence: 99%

Simultaneous Testing of Grouped Hypotheses: Finding Needles in Multiple Haystacks

Cai

Sun

2009

Journal of the American Statistical Association

107

112

View full text Add to dashboard Cite

In large-scale multiple testing problems, data are often collected from heterogeneous sources and hypotheses form into groups that exhibit different characteristics. Conventional approaches, including the pooled and separate analyses, fail to efficiently utilize the external grouping information. We develop a compound decision theoretic framework for testing grouped hypotheses and introduce an oracle procedure that minimizes the false nondiscovery rate subject to a constraint on the false discovery rate. It is shown that both the pooled and separate analyses can be uniformly improved by the oracle procedure. We then propose a data-driven procedure that is shown to be asymptotically optimal. Simulation studies show that our procedures enjoy superior performance and yield the most accurate results in comparison with both the pooled and separate procedures. A real-data example with grouped hypotheses is studied in detail using different methods. Both theoretical and numerical results demonstrate that exploiting external information of the sample can greatly improve the efficiency of a multiple testing procedure. The results also provide insights on how the grouping information is incorporated for optimal simultaneous inference.

show abstract

Section: Discussionmentioning

confidence: 99%

Simultaneous Testing of Grouped Hypotheses: Finding Needles in Multiple Haystacks

Cai

Sun

2009

Journal of the American Statistical Association

107

112

View full text Add to dashboard Cite

show abstract

“…Storey (2007) proposed the optimal discovery procedure (ODP) based on a shrinkage test statistic that maximizes the expected number of true positives (ETP) for a fixed level of the expected number of false positives (EFP). The result of Storey (2007) on the ODP follows directly from the theorem of Spjøtvoll (1972) by choosing appropriate f 0t and f t . In the setting of the present article, both Spjøtvoll's optimal procedure and Storey's ODP procedure depend on unknown functions that are not estimable from the data.…”

Section: Connection To Work Ofmentioning

confidence: 99%

“…We provide more numerical comparisons in Section 5, and show that the z value oracle procedure yields a lower mFNR level than the p value oracle procedure on all possible configurations of alternative hypotheses, with the difference significant in many cases. Spjøtvoll (1972) and Storey (2007) An "optimal" multiple-testing procedure was introduced by Spjøtvoll (1972 …”

Section: The Oracle Procedures Based On the P Valuesmentioning

confidence: 99%

“…The formulation of our testing procedure (6) has two advantages over the procedures of Spjøtvoll (1972) and Storey (2007). First, we can form good estimates (Jin and Cai 2007) from the data for the unknown functions in the oracle test statistic that we propose.…”

Section: Connection To Work Ofmentioning

confidence: 99%

“…First, we can form good estimates (Jin and Cai 2007) from the data for the unknown functions in the oracle test statistic that we propose. Second, we specify the optimal cutoff for each mFDR level, which was not discussed by Spjøtvoll (1972) or Storey (2007). These advantages greatly facilitate the development of the adaptive procedure (10), where a consistent cutoff (relative to the oracle cutoff) is suggested for each test level based on a simple step-up procedure.…”

Section: Connection To Work Ofmentioning

confidence: 99%

See 2 more Smart Citations

Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control

Sun

Cai

2007

Journal of the American Statistical Association

268

421

View full text Add to dashboard Cite

We develop a compound decision theory framework for multiple-testing problems and derive an oracle rule based on the z values that minimizes the false nondiscovery rate (FNR) subject to a constraint on the false discovery rate (FDR). We show that many commonly used multiple-testing procedures, which are p value-based, are inefficient, and propose an adaptive procedure based on the z values. The z value-based adaptive procedure asymptotically attains the performance of the z value oracle procedure and is more efficient than the conventional p value-based methods. We investigate the numerical performance of the adaptive procedure using both simulated and real data. In particular, we demonstrate our method in an analysis of the microarray data from a human immunodeficiency virus study that involves testing a large number of hypotheses simultaneously.

show abstract

Simultaneous Inference

Geller

O’Brien²

2005

Encyclopedia of Biostatistics

View full text Add to dashboard Cite

This covers statistical procedures to assess either more than one parameter or one parameter repeatedly during the course of a study. It includes multiple comparisons, multiple endpoints, sequential methods, and longitudinal data analysis. A common approach is to control the per‐family error rate (expected number of false positive conclusions per study) or the experimentwise error rate (probability of at least one false positive). Examples are given, with particular reference to problems of data monitoring of clinical trials.

show abstract

On the Optimality of Some Multiple Comparison Procedures

Cited by 85 publications

References 0 publications

Simultaneous Testing of Grouped Hypotheses: Finding Needles in Multiple Haystacks

Simultaneous Testing of Grouped Hypotheses: Finding Needles in Multiple Haystacks

Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control

Simultaneous Inference

Contact Info

Product

Resources

About