Optimal testing of multiple hypotheses with common effect direction

Bittman, Richard M.; Romano, Joseph P.; Vallarino, Carlos; Wolf, Michael

doi:10.1093/biomet/asp006

Cited by 29 publications

(49 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, even if the individual tests are constructed in some optimal manner, multiple testing procedures obtained by the closure method may in fact be inadmissible. This finding was previously obtained in Bittman et al (2009) in a specific context. In our analysis, the notion of consonance becomes pertinent.…”

Section: Introductionsupporting

confidence: 87%

See 1 more Smart Citation

Consonance and the Closure Method in Multiple Testing

Romano¹,

Shaikh²,

Wolf³

2011

The International Journal of Biostatistics

View full text Add to dashboard Cite

Consider the problem of testing s null hypotheses simultaneously. In order to deal with the multiplicity problem, the classical approach is to restrict attention to multiple testing procedures that control the familywise error rate (FWE). The closure method of Marcus et al. (1976) reduces the problem of constructing such procedures to one of constructing single tests that control the usual probability of a Type 1 error. It was shown by Sonnemann ( , 2008) that any coherent multiple testing procedure can be constructed using the closure method. Moreover, it was shown by Sonnemann and Finner (1988) that any incoherent multiple testing procedure can be replaced by a coherent multiple testing procedure which is at least as good. In this paper, we first show an analogous result for dissonant and consonant multiple testing procedures. We show further that, in many cases, the improvement of the consonant multiple testing procedure over the dissonant multiple testing procedure may in fact be strict in the sense that it has strictly greater probability of detecting a false null hypothesis while still maintaining control of the FWE. Finally, we show how consonance can be used in the construction of some optimal maximin multiple testing procedures. This last result is especially of interest because there are very few results on optimality in the multiple testing literature.

show abstract

Section: Introductionsupporting

confidence: 87%

“…Proof: It follows from Lemma 1 in Bittman et al (2009) that ϕ Λ,Λ is the most powerful test for testing h against h , among level αtests φ that also satisfy φ(x) = 0 if x ∈ R c . Let β Λ,Λ be its power against the alternative h .…”

mentioning

confidence: 99%

Consonance and the Closure Method in Multiple Testing

Romano¹,

Shaikh²,

Wolf³

2011

The International Journal of Biostatistics

View full text Add to dashboard Cite

show abstract

“…It should be emphasized that ensuring an MTP to be consonant is also important from both interpretive and mathematical statistics viewpoint (Bittman et al, 2009). However, consonance has not yet received as much attention and discussion as coherence.…”

Section: Introductionmentioning

confidence: 99%

“…An MTP is consonant if rejecting hypothesis H i will result in rejecting at least one of H j which is implied by H i . As pointed out by Bittman et al (2009), a nonconsonant or dissonant testing procedure can leave the statistician in a difficult situation when explaining the results of a study. For example, consider a randomized experiment for testing the efficacy of a drug versus a placebo with two primary endpoints: reduction in headaches and reduction in muscle pain.…”

Section: Introductionmentioning

confidence: 99%

“…The lack of consonance makes interpretation awkward. Bittman et al (2009) also argued from a mathematical statistics viewpoint that dissonance is undesirable in that it results in decreased ability to reject false null hypotheses. Sonnemann (2008) [the translated version of Sonnemann (1982)] presented a simple method, which will not reject H i if none of its supersets are rejected, to adjust dissonant procedure to be consonant.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

General solutions to consistency problems in multiple hypothesis testing

2010

View full text Add to dashboard Cite

Coherence and consonance are the two important concepts to ensure consistent conclusions drawn from multiple tests. Unlike coherence, consonance of a multiple testing procedure (MTP) has not yet received much attention. Although most of the MTPs used in practice are consonant, cases still exist that dissonant tests have to be used due to their unique advantages, for example, the likelihood ratio tests and sum tests. Consonance adjustments are necessary for such tests to avoid difficulty in interpretation. Moreover, in terms of detecting elementary hypotheses, a consonant test procedure is uniformly more powerful than the corresponding dissonant one. In this paper, several general methods using the partitioning principle to construct (strongly) consonant and strongly coherent MTPs are proposed. As a result, any dissonant (but coherent) multiple level α test can be improved by a consonant one without any cost in terms of inferences on elementary hypotheses.

show abstract

An evaluation of methods for testing hypotheses relating to two endpoints in a single clinical trial

Glimm

Whitehead

et al. 2012

Pharmaceutical Statistics

View full text Add to dashboard Cite

The issues and dangers involved in testing multiple hypotheses are well recognised within the pharmaceutical industry. In reporting clinical trials, strenuous efforts are taken to avoid the inflation of type I error, with procedures such as the Bonferroni adjustment and its many elaborations and refinements being widely employed. Typically, such methods are conservative. They tend to be accurate if the multiple test statistics involved are mutually independent and achieve less than the type I error rate specified if these statistics are positively correlated. An alternative approach is to estimate the correlations between the test statistics and to perform a test that is conditional on those estimates being the true correlations. In this paper, we begin by assuming that test statistics are normally distributed and that their correlations are known. Under these circumstances, we explore several approaches to multiple testing, adapt them so that type I error is preserved exactly and then compare their powers over a range of true parameter values. For simplicity, the explorations are confined to the bivariate case. Having described the relative strengths and weaknesses of the approaches under study, we use simulation to assess the accuracy of the approximate theory developed when the correlations are estimated from the study data rather than being known in advance and when data are binary so that test statistics are only approximately normally distributed.

show abstract

Optimal testing of multiple hypotheses with common effect direction

Cited by 29 publications

References 14 publications

Consonance and the Closure Method in Multiple Testing

Consonance and the Closure Method in Multiple Testing

General solutions to consistency problems in multiple hypothesis testing

An evaluation of methods for testing hypotheses relating to two endpoints in a single clinical trial

Contact Info

Product

Resources

About