Multilevel meta-analysis of single-subject experimental designs: A simulation study

Ugille, Maaike; Moeyaert, Mariola; Beretvas, S. Natasha; Ferron, John M.; Noortgate, Wim Van den

doi:10.3758/s13428-012-0213-1

Cited by 51 publications

(68 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model presented by Huitema and McKean (2000) has been frequently used in single-case methodological studies (e.g., Beretvas & Chung, 2008a;Ugille, Moeyaert, Beretvas, Ferron, & Van den Noortgate, 2012), given that it considers several aspects, such as the serial relation of data and the possibility of different types of effects, as well as trends. However, this model represents continuous data, which may not always be available in single-case studies in which count data (e.g., numbers of behaviors) are used.…”

Section: Data Generationmentioning

confidence: 99%

Assigning and combining probabilities in single-case studies: A second study

Manolov

Solanas

2013

Behav Res

View full text Add to dashboard Cite

The present study builds on a previous proposal for assigning probabilities to the outcomes computed using different primary indicators in single-case studies. These probabilities are obtained by comparing the outcome to previously tabulated reference values, and they reflect the likelihood of the results in the case that no intervention effect is present. In the present study, we explored how well different metrics are translated into p values in the context of simulation data. Furthermore, two published multiple-baseline data sets were used to illustrate how well the probabilities might reflect the intervention effectiveness, as assessed by the original authors. Finally, the importance of which primary indicator would be used in each data set to be integrated was explored; two ways of combining probabilities were used: a weighted average and a binomial test. The results indicated that the translation into p values worked well for the two nonoverlap procedures, with the results for the regression-based procedure diverging due to some undesirable features of its performance. These p values, when either taken individually or combined, were well aligned with effectiveness for the real-life data. These results suggest that assigning probabilities can be useful for translating the primary measure into the same metric, using these probabilities as additional evidence of the importance of behavioral change, complementing visual analysis and professionals' judgments.

show abstract

Section: Data Generationmentioning

confidence: 99%

Assigning and combining probabilities in single-case studies: A second study

Manolov

Solanas

2013

Behav Res

View full text Add to dashboard Cite

show abstract

“…Data were generated using Monte Carlo methods via the following model, presented by Huitema and McKean (2000), and used previously in other SCD simulation studies (e.g., Beretvas & Chung, 2008a;Ferron & Sentovich, 2002;Ugille et al, 2012):…”

Section: Methodsmentioning

confidence: 99%

“…Using the model presented above, ten three-tier MBD data sets (k = 10) were simulated for each iteration, and later integrated quantitatively. In previous simulation studies related to single-case meta-analysis (Owens & Ferron, 2012;Ugille et al, 2012) k = 10 was also one of the conditions studied. However, given that in those studies the estimation of effects was the object, k was more relevant than in the present study where weighting strategies are being compared.…”

Section: Insert Table 1 About Herementioning

confidence: 99%

“…Finally, it has been stressed that meta-analysis and the assessment of internal and external 1 We chose the term single-case designs (SCD) in order to be consistent with the labeling used in the articles recently published in this journal by, for instance, Baek and Ferron (2013), Shadish and Sullivan (2011), Shadish, Rindskopf, Hedges, and Sullivan (2013), although these designs are also referred to as single-subject experimental designs (e.g., Ugille, Moeyaert, Beretvas, Ferron, & Van den Noortgate, 2012). In any case, SCD are experimental in nature and not to be confused with case studies (Blampied, 2000).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Weighting strategies in the meta-analysis of single-case studies

2014

View full text Add to dashboard Cite

Establishing the evidence base of interventions taking place in areas such as psychology and special education is one of the research aims of single-case designs in conjunction with the aim of improving the well-being of participants in the studies. The scientific criteria for solid evidence focus on the internal and external validity of the studies, and for both types of validity, replicating studies and integrating the results of these replications (i.e., meta-analyzing) is crucial. In the present study we deal with one of the aspects of meta-analysis, namely the weighting strategy used when computing an average effect size across studies. Several weighting strategies suggested for single-case designs are discussed and compared in the context of both simulated and real-life data. The results indicated that there are no major differences between the strategies and, thus we consider that it is important to choose weights with a sound statistical and methodological basis, while scientific parsimony is another relevant criterion. More empirical research and conceptual discussion are warranted regarding the optimal weighting strategy in single-case designs, alongside investigation of the optimal effect size measure in these types of design.Key words: single-case designs; meta-analysis; weight; effect size 3 The evidence-based movement has now been salient for several years in a variety of disciplines including psychology (APA Presidential Task Force on Evidence-Based Practice, 2006), medicine (Sackett, Rosenberg, Gray, Hayness, & Richardson, 1996), and special education . In this context, single-case designs ( Sidman, 1960) and, apart from using random assignment of conditions to measurement times, it is also favored by replication of the behavioral change contiguous with the change in conditions (Kratochwill et al., 2013;Wolery, 2013). On the other hand, replication is also related to generalization (Sidman, 1960), which benefits from research synthesis and metaanalysis. In that sense, the evidence-based movement has also paid attention to the metaanalytical integration of replications or studies on the same topic (Beretvas & Chung, 2008b;Jenson, Clark, Kircher, & Kristjansson, 2007). The quantitative integration is deemed especially useful when moderator variables are included in the meta-analyses (Burns, 2012;Wolery, 2013).Finally, it has been stressed that meta-analysis and the assessment of internal and external 1 We chose the term single-case designs (SCD) in order to be consistent with the labeling used in the articles recently published in this journal by, for instance, Baek and Ferron (2013) ). The present study represents an effort to discuss and obtain evidence regarding the metaanalysis of single-case studies; its focus is on weighting strategies rather than on the effect size measures that summarize the results. In that sense, it should be stressed that we do not advocate here for or against specific procedures for SCD data analysis. We consider that, while the debate on the optimal analytical...

show abstract

“…Nevertheless, their performance for estimating variances is less than optimal unless the series are long (Moeyaert, Ugille, Ferron, Beretvas, & Van Den Noortgate, 2013;Ugille, Moeyaert, Beretvas, Ferron, & Van den Noortgate, 2012). Additionally, conducting the analysis and interpreting the results (most frequently done in relation to statistical rather than to clinical significance) is not straightforward and requires a certain amount of statistical knowledge and training.…”

mentioning

confidence: 99%