Robustness of the Analysis of Variance, the Welch Procedure and a Box Procedure to Heterogeneous Variances

Abstract

Numerous studies have documented the robustness of t and F to heterogeneous variances under the restricted condition of equal n's. Likewise, the distortion of 01. in the presence of unequal n's and variances has been demonstrated in both mathematical and empirical studies. Several investigations, however, have shown the Welch technique to be robust to this disturbing situation in the two group case. The present study was addressed to the k group AOV situation. Monte Carlo methods were employed to contrast several procedures with respect to a.) control over Type I errors and b.) power. Results indicate that the generalized Welch technique may be substituted for the AOV when variances are heterogenEous and n's are unequal.A TEST OF THE null hypothesis, H 0 : fJ. 1 =fJ. 2 = ... (J.k is generally conducted through analysis of variance (AOV) procedures. With K independent samples, the derivation assumes that the population variances are equal. In practice, however, the condition of homogeneous variances cannot always bernet. The effect of heterogeneous variances on the probability of a Type I error has been the subject ofboth J:IRthematical and empirical studies. As the review below will demonstrate, the effect of heterogeneous variances is generally not serious when n's are equal and of reasonable size, but when n's are unequal protection against Type I errors rapidly deteriorates.

Comparisons of Treatments After an Analysis of Variance in Ecology

“…-Our simulations and those ofKohr and Games (1974), Tomarken and Serlin (1986), and Wilcox et al (1986), indicate that the standard ANOV A F may be severely affected by variance heterogeneity. Unweighted-means ANOV A is even more affected (Milligan et al 1987).…”

“…For extremely unequal sample sizes (max/min ratio = 4), W may be too liberal; in fact neither test is robust to some patterns of variance heterogeneity, especially for many treatments (m > 4). However both Kohr and Games (1974) and Tomarken and Serlin ( 1986) found that W is robust for three or four treatments when the max/min ratio of sample sizes was 3. Clearly, very unequal sample sizes should be avoided.…”

Optimum Significance Levels for Multistage Comparison Procedures

“…For the normal case this result was already noted by Tukey in Chapter 30 of [6]. The levels ak are called apparent significance levels by Tukey [9], k-mean significance levels by Duncan [2], and nominal levels by Einot and Gabriel [3].…”

“…No exact test of homogeneity is then available; however a test proposed by Welch [10] seems to provide a highly satisfactory approximate test. (See Kohr and Games [6] for a comparison with several other tests.) The values of a 2 , • • • , as defined in Sections 4, 5 and 6 are then no longer exact and in fact depend slightly on the true values of the variance ratios.…”

Tests for Treatment Group Equality When Data are Nonnormal and Heteroscedastic

“…Many authors have highlighted available procedures for analyzing data that violate either the assumption of normality or the assumption of variance homogeneity. Brown and Forsythe (1974), Kohr and Games (1974) others demonstrated the general effectiveness (i.e., Type I error control) of Welch's (1938Welch's ( , 1951 two-sample and omnibus test statistics with heterogeneous variances. In addition, Keselman, Cribbie and Zumbo (1997), Wilcox (1995;, Yuen and Dixon (1973), and Zimmerman and Zumbo (1993a), among many others, have demonstrated the effectiveness of several alternatives to traditional parametric tests that can be used with nonnormal data, including nonparametric test statistics and tests with robust estimators (e.g., trimmed means).…”