Interpreting Effect Sizes of Education Interventions

Abstract

Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.

Effectiveness of Active Learning that Combines Physical Activity and Math in Schoolchildren: A Systematic Review

“…22,[29][30][31][36][37][38] When statistically significant, 5 of these studies had an ES > 0.2 (0.42-4.7) 22,29,31,36,38 which has been classified as large in the context of field-based education. 27,28 Two of the studies did not report statistically significant improvements in math scores, 30,37 although one did report a significant effect on sub-group analysis, with the lowest math performers experiencing a large, significant improvement compared to control (ES: 0.62; p < .005). 30 Because it is expected that primary/elementary school children will learn and improve math performance over time, the magnitude of the effect is an important consideration because any benefit needs to be weighed against cost, programming and available school resources.…”

“…27,28 The currently accepted ES of Cohen (small [0.2], medium [0.5], and large [0.8]) 25 are ''large'' in the context of field-based education interventions. 27,28 Therefore, the following ES standards (small [<0.05]; medium [0.05 < 0.2]; large [≥0.2]) have been used to interpret the academic outcomes for this review. The large ES has also been supported by a central source of scientific evidence, the What Works Clearinghouse.…”

“…Interventions that have a large ES and low cost are anticipated to be more scalable, and therefore have the capacity to reach a wider proportion of students. 27 Few studies examined other benefits of the intervention to the participants. The 3 studies that investigated change in BMI generally reported positive results, although the study conducted in low-income families was confounded by a nutrition intervention delivered concurrently.…”

“…• Measuring the cost and scalability of physically active lessons. 27 This may involve comparing the cost and benefits of utilizing existing classroom teachers, or recruiting PE specialists. 49 Administrators could also consider accessing existing programs and/or for resources on physically active lessons with the support available from third-party specialists.…”

Does attending a deeper learning school promote student motivation, engagement, perseverance, and achievement?

“…Study constructs were transformed into Rasch scores by AIR researchers according to the Rasch rating scale model (Andrich, 1978;Wright & Masters, 1982) using WINSTEPS (Linacre, 2005). 6 Kraft (2018) notes that effects that are considered small by Cohen's (1988) standards may be large in field-based school intervention studies. Although these effect sizes may be considered small (and this was not an intervention study), they may also represent important and measurable differences between students who experienced DL opportunities and those who did not.…”