Null hypothesis significance testing has dominated quantitative research in education and psychology. However, the statistical significance of a test as indicated by a p-value does not speak to the practical significance of the study. Thus, reporting effect size to supplement p-value is highly recommended by scholars, journal editors, and academic associations. As a measure of practical significance, effect size quantifies the size of mean differences or strength of associations and directly answers the research questions. Furthermore, a comparison of effect sizes across studies facilitates meta-analytic assessment of the effect size and accumulation of knowledge. In the current comprehensive review, we investigated the most recent effect size reporting and interpreting practices in 1,243 articles published in 14 academic journals from 2005 to 2007. Overall, 49% of the articles reported effect size-57% of which interpreted effect size. As an empirical study for the sake of good research methodology in education and psychology, in the present study we provide an illustrative example of reporting and interpreting effect size in a published study. Furthermore, a 7-step guideline for quantitative researchers is also summarized along with some recommended resources on how to understand and interpret effect size. Burnham, & Thompson, 2000;Kirk, 1996;Plucker, 1997;Robinson & Levin, 1997;Thompson, 1998bThompson, , 1999cThompson, , 1999dThompson & Snyder, 1997). The effectiveness of this recommendation is worthy of a methodological review. Thus, the purpose of the present study was to investigate the effect size reporting and interpreting practices in academic journals in education and psychology areas. To build the present study into a solid theoretical framework, we review the problems and misuses of NHST, and we emphasize the importance of effect size and its relationship with NHST, confidence intervals, and meta-analytical assessment of effect size. An illustrative example of reporting and interpreting effect size in a published study is also presented. To facilitate quantitative researchers' use of effect size, we recommend resources on how to understand and use effect size, and we provide a seven-step guideline for quantitative researchers to conclude the study.
Keywords
NHSTNHST is a traditional and popular approach to make statistical inference about research questions (Anderson et al., 2000). It is considered to be an objective, scientific procedure for knowledge accumulation (Kirk, 1996). It frames research questions in terms of two contrasting statistical hypotheses. For instance, when the purpose is to examine the effect of a treatment, the null hypothesis states that "the experimental group and the control group are not different with respect to [a specified property of interest] and that any difference found between their means is due to sampling fluctuation" (Carver, 1978, p. 381), whereas the alternative hypothesis states the opposite for a two-tailed test of mean difference as population parameter. When...