Empirically Derived Single-Case-Design Effect Size Distributions of Engagement and Challenging Behavior in Early Childhood Research

Abstract

The purpose of this study is to present a set of empirically derived effect size distributions in order to provide field-based benchmarks for assessing the relative effects of interventions aimed at reducing challenging behavior or increasing engagement for young children with and without disabilities. We synthesized 192 single-case designs that represented data from 162 individuals and nine groups of individuals (e.g., classes) in 53 reports. We generated distributions of standardized mean difference and log-response ratio using 197 effect sizes for engagement and 146 effect sizes for challenging behavior. We examined benchmarks as a function of publication, disability, dependent variable (primary/secondary), and functional relation status and reported distributions separately by engagement and challenging behavior. Overall, the effect size distributions suggest considerable variability in the magnitude of change associated with school-based interventions for engagement and challenging behavior assessed in the context of single-case designs. Data suggest that expected magnitude of change for engagement and challenging behavior interventions may be difficult to predict and that study and effect size characteristics influence the variability of the distributions. Our results have direct implications for researchers relative to assessing the efficacy of interventions aimed at reducing challenging behavior or increasing engagement in young children with and without disabilities.

A comparison of parental attention and preferred items during tummy time: A consecutive controlled case series evaluation

“…However, both formats of intervention implementation effectively increased head elevation and reduced negative vocalizations. It should be noted that our effectsize estimates are lower for both negative vocalizations and head elevation than are the median effect-size benchmarks for challenging behavior and engagement identified by Chow et al (2023). Third, based on outcomes of the Kruskal-Wallis test, there was no significant difference between the preferred item and parental attention conditions for negative vocalizations (p = 0.52, specified p < .05) and there was a significant difference between preferred item and parental attention conditions for head elevation (p = 0.032, specified p < .05).…”

“…Estimation methods are based on restricted maximum likelihood, and both α 1 and ffiffiffiffiffiffiffiffiffiffiffiffiffiffi τ 2 þ σ 2 p will remain unbiased when intervention effects are heterogeneous across participants and when the standard errors are not normally distributed (Valentine et al, 2016). The effect-size outcomes were then compared with the benchmarks that were identified by Chow et al (2023), which found a median effect size of 1.16 for challenging behavior and 1.52 for engagement. Last, we tested whether there were significant differences between the preferred item and parental attention conditions and between remote and in-person delivery of intervention for both negative vocalizations and head elevations using the nonparametric Kruskal-Wallis test (Breslow, 1970).…”

Single-case design effect-size distributions: Association with procedural parameters.

“…We conducted an electronic search via APA PsycInfo, PubMed, and ProQuest Dissertations and Theses. This article describes a subset of data from a larger project (J. C. Chow, 2023; Ledford, Eyler, et al, 2023); specific search terms and strategies are described in that article and are available via Open Access Materials (Ledford, Eyler, et al, 2023).…”

“…Given the relative recency of development and use of meaningful and viable single-case effect sizes, single-case researchers do not have as much information about the magnitude of behavior change likely in a given context. However, recent work has been done to establish both intervention-specific effect-size estimates (e.g., via meta-analyses; McClain et al, 2021; Samudre et al, 2023) and broader benchmarks that are not intervention-specific (J. C. Chow, 2023; Pustejovsky et al, 2023).…”

Introduction to the New Editors of Exceptional Children: The Editorial Vision of the New Team

“…Finally, Chow and colleagues (2023) present a set of empirically derived effect-size distributions to suggest field-based benchmarks for assessing relative effects of interventions aimed at increasing engagement or reducing challenging behavior for young children with and without disabilities. They found considerable variability in the magnitude of change associated with school-based interventions across a synthesis of single-case-design research studies.…”