Comparing Visual and Statistical Analysis in Single-Case Studies Using Published Studies

Harrington, Magadalena; Velicer, Wayne F.

doi:10.1080/00273171.2014.973989

Cited by 111 publications

(98 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The two models fitted to the completed data did significantly impact the RBESEs of the estimated effects, but not the RBs or RMSEs. Our results from fitting a simplified model to completed data are consistent with Harrington and Velicer (2015), who reported biased estimates of the error variance if autocorrelation was not accounted for. Specifically, negative autocorrelations that were unaccounted for resulted in overestimation of the error variance, and positive autocorrelations unaccounted for resulted in underestimation of the error variance (Harrington & Velicer, 2015).…”

Section: Discussionsupporting

confidence: 89%

“…Fourth, only positive autocorrelations were investigated, yet negative autocorrelations have been reported in empirical SCED studies Boldface β 2 (level shift) denotes that unacceptable estimates were uncovered under some manipulated conditions. * p < .05 and η 2 ≥ .06 (Harrington & Velicer, 2015;Parker et al, 2005;Shadish & Sullivian, 2011). For example, Shadish and Sullivan's review uncovered lag-1 autocorrelations ranging from − .93 to .79.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Dealing with missing data by EM in single-case studies

Chen

Feng

et al. 2019

Behav Res

View full text Add to dashboard Cite

Single-case experimental design (SCED) research plays an important role in establishing and confirming evidence-based practices. Due to multiple measures of a target behavior in such studies, missing information is common in their data. The expectation-maximization (EM) algorithm has been successfully applied to deal with missing data in between-subjects designs, but only in a handful of SCED studies. The present study extends the findings from Smith, Borckardt, and Nash (2012) and Velicer and Colby (2005b, Study 2) by systematically examining the performance of EM in a baseline-intervention (or AB) design under various missing rates, autocorrelations, intervention phase lengths, and magnitudes of effects, as well as two fitted models. Three indicators of an intervention effect (baseline slope, level shift, and slope change) were estimated. The estimates' relative bias, root-mean squared error, and relative bias of the estimated standard error were used to assess EM's performance. The findings revealed that autocorrelation impacted the estimates' qualities most profoundly. Autocorrelation interacted with missing rate in impacting the relative bias of the estimates, impacted the root-mean squared error nonlinearly, and interacted with the fitted model in impacting the relative bias of the estimated standard errors. A simpler model without autocorrelation can be used to estimate baseline slope and slope change in time-series data. EM is recommended as a principled method to handle missing data in SCED studies. Two decision trees are presented to assist researchers and practitioners in applying EM. Emerging research directions are identified for treating missing data in SCED studies.

show abstract

Section: Discussionsupporting

confidence: 89%

mentioning

confidence: 99%

Dealing with missing data by EM in single-case studies

Chen

Feng

et al. 2019

Behav Res

View full text Add to dashboard Cite

show abstract

“…However, note that we do not propose to replace visual analysis with the sole use of nonparametric CIs for ESs. We concur with the general consensus in the field of single-case research that visual and statistical analysis are complementary and in most cases should be used together to corroborate the conclusions and to increase the acceptability by the wider scientific community (e.g., Bulté & Onghena, 2012;Busk & Marascuilo, 1992;Harrington & Velicer, 2015;Tate et al, 2013).…”

Section: Introductionsupporting

confidence: 56%

“…The main advantage of visual analysis is that aspects of SCE data such as level, trend, variability, immediacy of the effect, and overlap can be assessed in a flexible way (Horner et al, 2005;Kratochwill, Levin, Horner, & Swoboda, 2014;Lane & Gast, 2014). However, visual analysis has been criticized for its lack of established formal decision guidelines which leaves the method vulnerable to subjectivity and inconsistency between researchers (e.g., Deprospero & Cohen, 1979;Fisch, 1998;Gibson & Ottenbacher, 1988;Harrington & Velicer, 2015;Ximenes, Manolov, Solanas, & Quera, 2009).…”

Section: Introductionmentioning

confidence: 99%

Confidence intervals for single-case effect size measures based on randomization test inversion

et al. 2016

View full text Add to dashboard Cite

In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for single-case effect size measures in the context of various single-case designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 -α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized single-case design. Additionally, we demonstrate how RTI can be extended to other types of single-case designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.

show abstract

“…The use of Cohen's benchmarks for single‐case designs has been put in doubt (Parker et al., ) and Harrington and Velicer () suggested referring to values between 1 and 2.5 as “medium effects.” However, they used the within‐case version and not the d ‐statistic created to be comparable to that obtainable from between‐group designs (Hedges, Pustejovsky, & Shadish, ).…”

mentioning

confidence: 99%