Combining One-Sided Binomial Tests

Abstract

Six methods of combining k independent binomial test statistics are compared with respect to their median significance levels, asymptotic relative efficiencies, and accuracy of null distribution approximations. The test statistics considered are the minimum significance level, Fisher's omnibus test, the likelihood ratio, an approximate likelihood ratio, the Mantel-Haenszel statistic, and the sum of chis. Fisher's test and the likelihood ratio perform relatively well for all alternative hypotheses. The Mantel-Haenszel procedure performs well when departures from the null hypotheses are similar for all tests. The minimum is sensitive to the situation in which most of the parameters assume the null value. The sum-of-chis procedure tends to assign too much weight to individual binomial statistics with large standard errors, whereas the approximate likelihood ratio assigns too much weight to statistics with small standard errors.

Statistical Reasoning in the Legal Setting

“…The usual (uncorrected) Fisher test yielded a p value of .0773, while Lancaster's (1949) corrected method (Louv and Littell 1986) yielded a p value of ,018. Considering that we are making a one-sided test and courts often require two-sided tests to yield significant results at about the .05 level, the difference in p values can be quite important.…”

“…overstratifying the data so that combination methods lose power 2. simply pooling several 2 x 2 tables into one large one (i.e.. ignoring the strata) 3. using a procedure, such as Fisher's summary statistic (-2 X In Li, where Li is the p value of the test in the ith strata and which is designed for continuous data) on discrete or count data without making an appropriate correction 4. insisting on a statistically significant difference in each strata rather than using a proper combination technique 5. making tests in each strata and rejecting the null hypothesis in a stratum based solely on the p value in the stratum without considering the multiple comparison aspect Courts often detect the first and second error but can hardly be expected to appreciate the technical nature of the third. The effect of discreteness on Fisher's procedure was investigated by Louv and Littell (1986), who combined binomial data sets. The data they used arose in Cooper v. The University of Texas at Dallas (1979) and concerned a change of sex discrimination in hiring faculty.…”

Statistical Issues Arising in Disparate Impact Cases and the Use of the Expectancy Curve in Assessing the Validity of Pre‐Employment Tests

“…The method of using the Gail-Simon test as a check that the odds ratios are qualitatively similar and then applying the CMH test (Gastwirth, 1988;Finkelstein & Levin, 2001) is easy to implement and avoids the problem of assessing whether the assumptions needed for the validity of the simple combined table hold. Other combination methods may also be appropriate but one needs to adjust some, e.g., Fisher's summary chi-squared statistic to account for the discrete nature of the data (Gastwirth, 1984;Louv & Littell, 1986).…”

A Statistical Analysis of Adverse Impact of Employer Decisions

“…However, it leaves open the question of whether the effects observed truly reflect age discrimination or are an artifact of defining the protected group to be people aged 40 and older. One way of looking at that question statistic The Mantel-Haenszel statistic is used in articles discussing discrimination in hiring under a model of two or more independent populations (Gastwirth 1984;Gastwirth and Greenhouse 1987;Louv and Littel 1986).…”