Realist randomised controlled trials: A new approach to evaluating complex public health interventions

Abstract

Randomized trials of complex public health interventions generally aim to identify what works, accrediting specific intervention 'products' as effective. This approach often fails to give sufficient consideration to how intervention components interact with each other and with local context. 'Realists' argue that trials misunderstand the scientific method, offer only a 'successionist' approach to causation, which brackets out the complexity of social causation, and fail to ask which interventions work, for whom and under what circumstances. We counter-argue that trials are useful in evaluating social interventions because randomized control groups actually take proper account of rather than bracket out the complexity of social causation. Nonetheless, realists are right to stress understanding of 'what works, for whom and under what circumstances' and to argue for the importance of theorizing and empirically examining underlying mechanisms. We propose that these aims can be (and sometimes already are) examined within randomized trials. Such 'realist' trials should aim to: examine the effects of intervention components separately and in combination, for example using multi-arm studies and factorial trials; explore mechanisms of change, for example analysing how pathway variables mediate intervention effects; use multiple trials across contexts to test how intervention effects vary with context; draw on complementary qualitative and quantitative data; and be oriented towards building and validating 'mid-level' program theories which would set out how interventions interact with context to produce outcomes. This last suggestion resonates with recent suggestions that, in delivering truly 'complex' interventions, fidelity is important not so much in terms of precise activities but, rather, key intervention 'processes' and 'functions'. Realist trials would additionally determine the validity of program theory rather than only examining 'what works' to better inform policy and practice in the long-term.

What is the level of evidence for the use of currently available technologies in facilitating the self-management of difficulties associated with ADHD in children and young people? A systematic review

“…It also means that it is difficult to control each arm of an RCT study design. It has therefore been suggested [78] that the integration of realist evaluation within an RCT design may be more appropriate for evidence-based medicine whereby "statistically significant benefits may be marginal in clinical practice" [79].…”

The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance

“…Increased use of RCTs reinforces this position, limiting the role of expertise to a technical exercise in reducing uncertainty rather than a practical quest to determine the most appropriate policies using a broad set of criteria (Sanderson 2004, p. 376). While philosophical debates continue over the status of RCT knowledge claims (Hammersley 2005;Chalmers 2005;Cartwright 2007;Bonell et al 2012;Marchal et al 2013), the rhetoric of those proposing greater use of RCTs often goes beyond a plea to merely add RCTs to the methodological toolkit of social researchers. The Cabinet Office report advocating RCTs adopts a clear stance from the outset:…”

Evaluation of occupational health interventions using a randomized controlled trial: challenges and alternative research designs

“…For instance, the order of intervention components may be altered or the intervention components may be tailored to a specific group of workers or to specific occupational health problems. If adjustments are made within clusters, it becomes difficult to establish which intervention components or what implementation processes contributed to the effectiveness or lack thereof of the intervention, a situation sometimes referred to as a 'black box' (19,20).…”