Assigning and combining probabilities in single-case studies: A second study

Abstract

The present study builds on a previous proposal for assigning probabilities to the outcomes computed using different primary indicators in single-case studies. These probabilities are obtained by comparing the outcome to previously tabulated reference values, and they reflect the likelihood of the results in the case that no intervention effect is present. In the present study, we explored how well different metrics are translated into p values in the context of simulation data. Furthermore, two published multiple-baseline data sets were used to illustrate how well the probabilities might reflect the intervention effectiveness, as assessed by the original authors. Finally, the importance of which primary indicator would be used in each data set to be integrated was explored; two ways of combining probabilities were used: a weighted average and a binomial test. The results indicated that the translation into p values worked well for the two nonoverlap procedures, with the results for the regression-based procedure diverging due to some undesirable features of its performance. These p values, when either taken individually or combined, were well aligned with effectiveness for the real-life data. These results suggest that assigning probabilities can be useful for translating the primary measure into the same metric, using these probabilities as additional evidence of the importance of behavioral change, complementing visual analysis and professionals' judgments.

Probability and Visual Aids for Assessing Intervention Effectiveness in Single-Case Designs

“…Regarding the probability-based approach (MRA), previous evidence suggested that it works well as a translation mechanism between nonoverlap indices (Manolov & Solanas, 2013b). The evidence presented here shows that the similarities extend to standardized difference indices, given the positive and high correlations obtained.…”

Outcome assessment and inference with the percentage of nonoverlapping data (PND) single-case statistic.

“…Using simulation methods, Manolov and Solanas (2008) found that PND was the statistic least affected by the presence of autocorrelation when compared to several other single-case effect size measures. Another study by Manolov and Solanas (2013) found that PND’s p values (estimated via simulation) were similarly unaffected by autocorrelation. PND is thus an attractive option for single-case investigators concerned about serial dependency in their data.…”

“…Because its sampling distribution is unknown, it is difficult to perform null hypothesis significance testing (Beretvas & Chung, 2008). PND p values can be estimated with Monte Carlo simulation techniques (Manolov & Solanas, 2013), though few applied researchers may be able or willing to use those computationally advanced methods. Without a defined sampling distribution, PND is a descriptive statistic whereas other effect size measures are inferential (Allison & Gorman, 1994).…”