Revision of a method quality rating scale for single-case experimental designs andn-of-1 trials: The 15-item Risk of Bias inN-of-1 Trials (RoBiNT) Scale

Abstract

Recent literature suggests a revival of interest in single-case methodology (e.g., the randomised n-of-1 trial is now considered Level 1 evidence for treatment decision purposes by the Oxford Centre for Evidence-Based Medicine). Consequently, the availability of tools to critically appraise single-case reports is of great importance. We report on a major revision of our method quality instrument, the Single-Case Experimental Design Scale. Three changes resulted in a radically revised instrument, now entitled the Risk of Bias in N-of-1 Trials (RoBiNT) Scale: (i) item content was revised and increased to 15 items, (ii) two subscales were developed for internal validity (IV; 7 items) and external validity and interpretation (EVI; 8 items), and (iii) the scoring system was changed from a 2-point to 3-point scale to accommodate currently accepted standards. Psychometric evaluation indicated that the RoBiNT Scale showed evidence of construct (discriminative) validity. Inter-rater reliability was excellent, for pairs of both experienced and trained novice raters. Intraclass correlation coefficients of summary scores for individual (experienced) raters: ICC(TotalScore) = .90, ICC(IVSubscale) = .88, ICC(EVISubscale) = .87; individual (novice) raters: ICC(TotalScore)= .88, ICC(IVSubscale) = .87, ICC(EVISubscale) = .93; consensus ratings between experienced and novice raters (ICC(TotalScore) = .95, ICC(IVSubscale) = .93, ICC(EVISubscale) = .93. The RoBiNT Scale thus shows sound psychometric properties and provides a comprehensive yet efficient examination of important features of single-case methodology.

Perspective training to treat anger problems after brain injury: Two case studies

“…Additionally, the design of our case study had limited control conditions, making it difficult to determine the effect of treatment. For instance, we only had two baseline assessments for the aggression questionnaire and one pre-intervention assessment for perspective-taking, which does not meet the minimum three times that is recommended for single-subject designs (Tate et al, 2013). Another aspect we were not able to control is the treatment they were receiving while participating in the perspectives-training group.…”

External memory aids for memory problems in people with multiple sclerosis: A systematic review

“…This review classifies SCEDs using the SCED scale (Tate et al, 2008), which includes AB designs, such as the Lincoln et al (2003) study. A revised classification system has since been developed, the RoBiNT Scale (Tate et al, 2013), which states that AB designs should not be classified as SCEDs due to the inability to determine cause and effect, with the absence of ABA reversal or multiple baseline designs. Therefore it should be noted that the included SCED (Lincoln et al, 2003) does not provide staggered baselines.…”

Further developments in summarising and meta-analysing single-case data: An illustration with neurobehavioural interventions in acquired brain injury

“…In contrast, Cattelani and colleagues (2010) stress that the limitations of group-design studies in terms of including participants with very different demographic characteristics, aetiology, site of brain damage, and so forth, hinder the assessment of how large the effect of a treatment would be for a particular patient. Some of these issues of uncontrolled factors affecting the certainty of the causal effect of interventions can be addressed in SCEDs, especially if recommendations on study conduct (Horner et al, 2005;Tate et al, 2013) are followed, including measures of maintenance and generalization (frequently missing according to Ylvisaker et al, 2007), social validity, and procedural fidelity (also rare according to Heinicke & Carr, 2014), among others. Thus, the combination of several methodological options and the integration of results across studies can prove to be very useful in the field.…”

“…Regarding the SCED structures being well-suited for building evidence-based interventions, Kratochwill et al ( , 2013 and Tate et al (2013) have emphasized the need for several transitions between phases with and without intervention and the importance of deciding the points of change in phase at random. Several possibilities exist for meeting the methodological criterion regarding the design structure: alternating treatments designs, reversal/withdrawal designs, MBDs, etc.…”

“…For designs in which only one AB comparison is performed (designs that are methodologically weaker; Tate et al, 2013), CV is set to 1 in order to use only the number of measurements available as a weight. It is also possible to stick to the more classic number of measurements available in all data sets from the study as a weight for the study effect size, as we did for the standardised version of the MPD and SLC procedures applied here.…”