Optimum Significance Levels for Multistage Comparison Procedures

Abstract

The framework for multistage comparison procedures in the present paper is roughly that introduced by Duncan and treated more fully by Tukey. In the present paper we consider the problem of finding the optimum allocation of nominal significance levels for successive stages. The optimum procedure we obtain when the number s of treatments is odd, and the compromise procedure we propose for evens, essentially agree with a procedure suggested by Tukey. The agreement is exact when s is even and close when s is odd. The results of the present paper apply among others to the problem of distinguishing normal distributions with known variances, multinomial distributions, Poisson distributions, and distributions in certain nonparametric settings. However, they do not apply exactly to the comparison of normal distributions with a common unknown variance. When the variances are completely unknown, the method applies in principle but faces the difficulty that no exact test is then available for testing the homogeneity of a set of means.

Erich Lehmann’s contributions to multiple decision making

“…In the so-called stepwise methods, the level for testing any hypothesis also depends on the outcomes of tests of other hypotheses. This paper (Lehmann and Shaffer, 1979) deals with multiple range procedures, one type of stepwise method applicable to comparisons of the effects of treatments. Some optimality conditions are proposed and a procedure derived that turns out to be almost identical to a widely-used procedure proposed by Tukey, who did not note any optimality properties.…”