The authors conducted a 30-year review (1969-1998) of the size of moderating effects of categorical variables as assessed using multiple regression. The median observed effect size (f(2)) is only .002, but 72% of the moderator tests reviewed had power of .80 or greater to detect a targeted effect conventionally defined as small. Results suggest the need to minimize the influence of artifacts that produce a downward bias in the observed effect size and put into question the use of conventional definitions of moderating effect sizes. As long as an effect has a meaningful impact, the authors advise researchers to conduct a power analysis and plan future research designs on the basis of smaller and more realistic targeted effect sizes.

Investigators in numerous organization studies disciplines are concerned about the low statistical power of moderated multiple regression (MMR) to detect effects of categorical moderator variables. The authors provide a theoretical approximation to the power of MMR. The theoretical result confirms, synthesizes, and extends previous Monte Carlo research on factors that affect the power of MMR tests of categorical moderator variables and the low power of MMR in typical research situations. The authors develop and describe a computer program, which is available on the Internet, that allows researchers to approximate the power of MMR to detect the effects of categorical moderator variables given user-input information (e.g., sample size, reliability of measurement). The approximation also allows investigators to determine the effects of violating certain assumptions required for MMR. Given the typically low power of MMR, researchers are encouraged to use the computer program to approximate power while planning their research design and methodology.

No abstract

repeated measures designs, sphericity criterion, multiple contrasts,

The Fisher–Pitman permutation test is an increasingly popular alternative to the ANOVA F test. As a test of equality of distributions, the permutation test is very attractive because it retains its stated test size without any distributional requirements. As a test of equality of location parameters, however, the permutation test retains its stated test size only under equality of all nuisance parameters. This homogeneity requirement is well known, but often overlooked. As a result, the permutation test is sometimes recommended when, in fact, the distributional requirements are not satisfied. This article examines the robustness of the permutation test to violation of the homogeneity requirement. In particular, the size of the Fisher–Pitman permutation test of equality of means is compared to the size of the normal theory F test in small samples when variances are unequal. Normally distributed populations having equal means are assumed. The size of the permutation test is found to be smaller than the size of the F test when the ratio of the harmonic to the arithmetic mean of the sample sizes is small and vice versa when the ratio is large (i.e. near 1). In either case, the difference between the sizes of the two tests is relatively small, except for extreme heterogeneity. The result is based on a comparison of the moments of the permutation and normal theory sampling distributions and is supported by the results of simulation studies.