The purpose of this research is to examine eight of the different methods for computing confidence intervals around alpha that have been proposed to determine which of these, if any, is the most accurate and precise. Monte Carlo methods were used to simulate samples under known and controlled population conditions. In general, the differences in the accuracy and precision of the eight methods examined were negligible in many conditions. For the breadth of conditions examined in this simulation study, the methods that proved to be the most accurate were those proposed by Bonett and Fisher. Larger samples sizes and larger coefficient alphas also resulted in better interval coverage, whereas smaller numbers of items resulted in poorer interval coverage.

Various tests to check the homogeneity of variance assumption have been proposed in the literature, yet there is no consensus as to their robustness when the assumption of normality does not hold. This simulation study evaluated the performance of 14 tests for the homogeneity of variance assumption in one-way ANOVA models in terms of Type I error control and statistical power. Seven factors were manipulated: number of groups, average number of observations per group, pattern of sample sizes in groups, pattern of population variances, maximum variance ratio, population distribution shape, and nominal alpha level for the test of variances. Overall, the Ramsey conditional, O'Brien, Brown-Forsythe, Bootstrap Brown-Forsythe, and Levene with squared deviations tests maintained adequate Type I error control, performing better than the others across all the conditions. The power for each of these five tests was acceptable and the power differences were subtle. Guidelines for selecting a valid test for assessing the tenability of this critical assumption are provided based on average cell size.

No abstract

This study was conducted to evaluate alternative analysis strategies for the metaanalysis method of reliability generalization when the reliability estimates are not statistically independent. Five approaches to dealing with the violation of independence were implemented: ignoring the violation and treating each observation as independent, calculating one mean or median from each study, selecting only one observation per study, and using a mixed-effects model. Monte Carlo methods were used to simulate samples under known and controlled population conditions. The results suggest that the type of approach does not have a noticeable impact on the accuracy of the reliability results but that researchers should be cautious when the intraclass correlation is relatively large. The simulations in this study also resulted in very poor confidence band coverage.

No abstract