Multiple-baseline studies are prevalent in behavioral research, but questions remain about how to best analyze the resulting data. Monte Carlo methods were used to examine the utility of multilevel models for multiplebaseline data under conditions that varied in the number of participants, number of repeated observations per participant, variance in baseline levels, variance in treatment effects, and amount of autocorrelation in the Level 1 errors. Interval estimates of the average treatment effect were examined for two specifications of the Level 1 error structure ( 2 I and first-order autoregressive) and for five different methods of estimating the degrees of freedom (containment, residual, between-within, Satterthwaite, and Kenward-Roger). When the Satterthwaite or Kenward-Roger method was used and an autoregressive Level 1 error structure was specified, the interval estimates of the average treatment effect were relatively accurate. Conversely, the interval estimates of the treatment effect variance were inaccurate, and the corresponding point estimates were biased.

Given the statistical bias that occurs when design effects of complex data are not incorporated or sample weights are omitted, this study calls for improvement in the dissemination of research findings based on complex sample data. Authors, editors, and reviewers need to work together to improve the transparency of published findings using complex sample data.

The purpose of this research is to examine eight of the different methods for computing confidence intervals around alpha that have been proposed to determine which of these, if any, is the most accurate and precise. Monte Carlo methods were used to simulate samples under known and controlled population conditions. In general, the differences in the accuracy and precision of the eight methods examined were negligible in many conditions. For the breadth of conditions examined in this simulation study, the methods that proved to be the most accurate were those proposed by Bonett and Fisher. Larger samples sizes and larger coefficient alphas also resulted in better interval coverage, whereas smaller numbers of items resulted in poorer interval coverage.