Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers.

Abstract

We describe convenient statistical procedures that will enable research consumers (e.g., professional psychologists, graduate students, and researchers themselves) to reach beyond the published conclusions and make an independent assessment of the reported results. Appropriately conceived contrasts accompanied by effect size estimates often allow researchers to address precise predictions that the authors of the published report may have ignored or abandoned prematurely. We describe the use of t, F, and Z to compute contrasts with different raw ingredients, and we review 3 effect size indices (Cohen's d, Hedges's g, and the Pearson r) and a way of displaying the magnitude of any effect size r. We also describe how to construct confidence limits for the obtained effect as well as its null-counternull interval.Following in the wake of Jacob Cohen's seminal work on the utility of effect sizes and the neglect of statistical power in psychological research (e.g., Cohen, 1962Cohen, , 1965Cohen, , 1969, there has been a surge of interest in these issues (e.g., Judd, McClelland, & Culhane, 1995). Cohen not only provided researchers with an elegant and useful discussion of effect sizes and power analysis (e.g., Cohen, 1969Cohen, , 1988, but he also pioneered in the empirical work showing that significance testing in psychology was conducted with a remarkably high risk of committing Type II

Anger in Pathological Gambling: Clinical, Psychopathological, and Personality Correlates

“…Next, we assessed whether sex is a predictive variable for the mean STAXI-2 scores within each group (clinical-control) with an ANCOVA (adjusted by age and duration of the disorder for the PG sample). Pearson's correlation coefficients were calculated to examine the linear association between STAXI-2, SCL-90-R, and TCI-R direct scores, and absolute values above .30 were considered for practical significance (Rosnow & Rosenthal, 1996). Finally, the predictive capacity of the STAXI-2 scores (independent variables or predictors) on the intensity of the disorder (dependent variables or criteria) for the PG sample was evaluated thorough multiple linear regression models (for quantitative criteria, SOGS total score) and negative binomial models (for criteria that represent counting, such as number of the DSM-IV criteria and the total number of games briefed by the patient) adjusted by sex and duration of the disorder.…”

Psychosocial treatments of behavior symptoms in dementia: a systematic review of reports meeting quality standards

“…Effect sizes based on mean differences and standard deviations were calculated for studies that reported sufficient information (Rosnow and Rosenthal, 1996). For randomized studies, the effect size was the difference between the treatment mean and the attention control mean divided by an estimate of standard deviation pooled from both groups.…”

Attentional bias for positive emotional stimuli: A meta-analytic investigation.

“…This conservative approach (Rosenthal, 1991) allows a better representation of the existing literature and was used only when it was possible to determine the direction of the effect through the provided descriptive statistics (15.22% of cases; for a similar procedure, see Bar-Haim et al, 2007). In cases in which the contrast of interest was not directly tested through a planned test (e.g., positive vs. neutral), but through a main effect test that included other modalities (e.g., main effect of valence: Positive, negative, or neutral), the effect size for the contrast of interest (i.e., positive vs. neutral) was extracted through the maximum possible contrast F method (MPC-F; Rosnow & Rosenthal, 1996). The method used to extract the effect size (i.e., F, t, p, and MPC-F) was entered as a control moderator and the between-groups homogeneity statistic revealed a significant effect (Q B ϭ 10.819, p ϭ .012).…”