Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors.

Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes.

“…This can be solved only by doing more primary studies. The good news is that power may be less of a problem in meta-analysis than in primary research (Osburn, Callender, Greener, & Ashworth, 1983;Sackett, Harris, & Orr, 1986;Spector & Levine, 1987). The reason is that the unit of analysis in primary research is usually an individual, but the unit in meta-analysis is a study-level effect size that is an aggregate of these primary units.…”

The effects of team training on team outcomes: A meta-analysis

“…A bare-bones meta-analysis is conservative because it includes no corrections for artifacts in the data beyond sampling error as additional corrections can provide inaccurate results from the analyses when using a small sample of studies (Spector & Levine, 1987). Because our study included only published, refereed studies, we constructed a funnel plot using each effect size and its associated sample size to determine whether the sample we used was subject to publication bias.…”

Messages as replications: Toward a message‐centered design strategy

“…We base this conclusion on a second Monte Carlo study (modelled after Spector and Levine, 1987).^ To understand the logic of the comparison, imagine 1000 respondents divided among 10 messages subjected to a common treatment, and analyzed in two different ways. One way is to consider the 10 messages as constituting 10 separate studies (as Allen et al did) and to assess messageby-treatment interaction by comparing the variance in the effect sizes for each message with the variance expected from sampling error; this is the Hunter-Schmidt procedure.…”

“…Briefly, the Monte Carlo study compared the power of the Hunter-Schmidt procedure and the power of the interaction F-test under a variety of conditions similar to those examined by Spector and Levine (1987). Where Spector and Levine (1987) generated Monte Carlo estimates of the power of Hunter et al's (1982) procedures to detect differences in underlying correlations, we substituted effect sizes \d) for correlations and also obtained estimates of the power of F. The important results for the present discussion are these: As in Spector and Levine's study, some common ways of applying the Hunter et al procedure involve de facto Type I error rates ranging between 15% and 40%, but whenever the Type I error rate for the Hunter et al procedure is brought under control, its power to detect variability of treatment effects across messages is identical to a corresponding i^-test with the same Type I error rate (a result that is consistent across study sizes and across varying amounts of variability).…”