This paper presents a unilevel and multilevel approach for the analysis and meta-analysis of singlecase experiments (SCEs). We propose a definition of SCEs and derive the specific features of SCEs' data that have to be taken into account when analyzing and meta-analyzing SCEs. We discuss multilevel models of increasing complexity and propose alternative and complementary techniques based on probability combining and randomization test wrapping. The proposed techniques are demonstrated with real-life data and corresponding R code.
KeywordsSingle-case experiment; Single-case experimental design; N-of-1 trial; Meta-analysis; Multilevel model; Probability combining; Randomization test; Permutation testCorresponding author: Patrick Onghena, KU Leuven -University of Leuven, Faculty of Psychology and Educational Sciences, Tiensestraat 102, BE-3000 Leuven, Belgium. E-mail: patrick.onghena@kuleuven.be.
3Many handbooks and courses in research methods and statistics seem to imply that groups have to be compared in order to conduct proper scientific research in the health sciences (see e.g., Jacobson, 2017;Moore, McCabe, & Craig, 2017;Peat, 2002). In those handbooks and courses, treatment effects are inferred by applying treatments to one or more experimental groups and by comparing their results to one or more control groups. The mantra is: "Take a random sample of patients and randomly assign these patients to treatment and control". In medicine, this mantra is consolidated in the randomized controlled trial being the gold standard (Kaptchuk, 2001; Sacket, Rosenberg, Gray, Haynes, & Richardson, 1996;Turner et al., 2012).The practical disadvantages and ethical concerns of this group-comparison approach are well documented (Carey & Stiles, 2016), but more importantly, questions regarding the epistemological status of the results, and therefore also the clinical relevance, can be raised (Molenaar, 2004;Onghena & Edgington, 2005;Onghena, 2007). Intra-patient variability differs fundamentally from inter-patient variability, and consequently there is a fundamental difference between the meaning of a treatment effect in group-comparison studies and the meaning of a treatment effect for an individual patient (Molenaar & Campbell, 2009;Velicer & Molenaar, 2013). The reckless generalization of group-comparison treatment effects to individual-patient treatment effects can be considered as a prime example of the well-known ecological fallacy (Harrington & Velicer, 2015;Onghena, 2016). Group-comparison treatment effects are not necessarily representative of individual-patient treatment effects; it might well be that the average pattern of group differences does not apply to any single patient involved in the study (Barlow, Nock, & Hersen, 2009;Gast & Ledford, 2014;Kazdin, 2011). As Sidman already observed in 1952: "It appears, then, that when different groups of subjects are used to obtain the points determining a functional relation, the mean curve does not provide the information necessary to make statements concerning the ...