On hypothesis testing with a partitioned random alternative

Abstract

It is common in statistical practice that one needs to make a choice among m + 1 mutually exclusive claims on distributions. When m = 1, it is done by the (traditional) hypothesis test. In this paper, a generalization to the case m > 1 is proposed. The fundamental difference with the case m = 1 is that the new alternative hypothesis is a partition of m multiple claims and is data-dependent. Data is used to decide which claim in the partition is to be tested as the alternative. Thus, a random alternative is involved. The conditional and overall type I errors of the proposed test are controlled at a given level, and this test can be used as a new solution for the general multiple test problem. Several classical problems, including the one-sample problem, model selection in multiple linear regression, and multi-factor analysis, are revisited, and new tests are provided correspondingly. Consequently, the famous two-sided t-test should be replaced by the proposed.