Sequential Bonferroni Methods for Multiple Hypothesis Testing with Strong Control of Family-Wise Error Rates I and II

Abstract

Sequential procedures are developed for simultaneous testing of multiple hypotheses in sequential experiments. Proposed stopping rules and decision rules achieve strong control of both family-wise error rates I and II. The optimal procedure is sought that minimizes the expected sample size under these constraints. Bonferroni methods for multiple comparisons are extended to sequential setting and are shown to attain an approximately 50% reduction in the expected sample size compared with the earlier approaches. Asymptotically optimal procedures are derived under Pitman alternative.

Stomata morphological traits in two different genotypes of Populus nigra L.

“…Differences among mean values were tested by post-hoc analyses using the Bonferroni's test (α = 0.05), which was considered the most appropriate due to the limited number of samples analyzed. Bonferroni methods for multiple comparisons are extended to sequential setting and have shown to attain an approximately 50% reduction in the expected sample size compared with earlier approaches (De & Baron 2012).…”

Sequential tests controlling generalized familywise error rates

“…Control of both error rates appears possible due to the flexibility of sequential designs, similarly to the singlehypothesis SPRT attaining both desired probabilities of Type I and Type II errors. A modification of this sequential procedure is proposed in [4], combining the ideas of Wald's SPRT and Holm-type stepwise testing. Improving the plain Bonferroni methods, this new algorithm requires a smaller expected sample size, reducing the overall expected costs of the experiment and at the same time controlling both familywise error rates.…”

“…We assume that each sampled unit i contributes to the total cost of an experiment regardless of how many components X ij (such as vital signs of patients or electronic measurements of manufactured parts) are recorded on unit i. This is quite common in many experiments (e.g., [4,11,15,26]). For example, in clinical trials, certain amount is budgeted for each participating patient, covering the cost of a treatment, service, insurance, incentive, and possibly, accommodation and transportation.…”

“…It is to be distinguished from the total number of recorded measurements X ij that is used in [1] as a sample size (and misinterpreted in [4,5]). …”

“…An adaptive multistage step-down procedure proposed in [1] controls the Type I familywise error rate, defined as the probability of rejecting at least one true hypothesis. Generalizing the concept of Wald's sequential probability ratio test (SPRT) to multiple hypotheses, [4] develops a testing procedure that controls both Type I and Type II familywise error rates in the strong sense. By analogy, the latter is defined as the probability of accepting at least one false null hypothesis.…”