Realist RCTs of complex interventions – An oxymoron

Marchal, Bruno; Westhorp, Gill; Wong, Geoff; Belle, Sara Van; Greenhalgh, Trisha; Kegels, Guy; Pawson, Ray

doi:10.1016/j.socscimed.2013.06.025

Cited by 133 publications

(133 citation statements)

References 15 publications

Supporting

Mentioning

125

Contrasting

Unclassified

Order By: Relevance

“…Increased use of RCTs reinforces this position, limiting the role of expertise to a technical exercise in reducing uncertainty rather than a practical quest to determine the most appropriate policies using a broad set of criteria (Sanderson 2004, p. 376). While philosophical debates continue over the status of RCT knowledge claims (Hammersley 2005;Chalmers 2005;Cartwright 2007;Bonell et al 2012;Marchal et al 2013), the rhetoric of those proposing greater use of RCTs often goes beyond a plea to merely add RCTs to the methodological toolkit of social researchers. The Cabinet Office report advocating RCTs adopts a clear stance from the outset:…”

Section: Plurality Of Evidencementioning

confidence: 99%

The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance

Pearce

Raman

2014

Policy Sci

View full text Add to dashboard Cite

Since 2012, a new movement of government departments, think tanks and highprofile individuals within the UK has sought to promote the increased usage of randomised controlled trials (RCTs) in public policy. They promote RCTs as an evidence-based corrective for the inappropriate certainties of experts. Recent government reports and public debate around this initiative are reviewed and analysed within a framework for epistemic governance: normative insights into how knowledge for policymaking should be understood and governed drawn from science and technology studies and the policy sciences. The legitimacy of RCT evidence within policymaking is found to rest on the recognition of three key features: (1) how multiple meanings of evidence limit generalisability, (2) ensuring a plurality of evidence inputs, including those from other forms of research and expertise, and (3) building institutions for governing the use of RCTs in the public interest. Producing evidence for policymaking is a hybrid activity that necessarily spans both science and politics. Presenting RCTs as naively neutral evidence of what policy interventions work is misleading. The paper concludes by calling for more work on how the new RCT movement might engage with its own history in social and policy research on the value of experiments for policymaking.

show abstract

Section: Plurality Of Evidencementioning

confidence: 99%

The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance

Pearce

Raman

2014

Policy Sci

View full text Add to dashboard Cite

show abstract

“…Er wordt sterk getwijfeld of een experimenteel onderzoek wel voldoende toelaat om inzicht te verwerven in de effectiviteit van complexe interventies (Hermans, 2014;Marchal et al, 2013;Otto et al, 2009). Het argument is dat effectstudies met een experimenteel design geen verklaring geven voor hoe en waarom de interventie in die praktijkcontext, met die doelgroep, die effecten bereikt (Pawson, 2013;Solmeyer & Constance, 2015).…”

Section: T H E O R E T I S C H O N D E R B O U W E N Va N S O C I a Lunclassified

“…Deze praktijkcontext en menselijke invloeden veranderen continu en zijn dus per definitie complex (Rogers, 2008). Experimentele designs beogen deze externe invloeden beheersbaar te maken (Marchal et al, 2013). Om voorspellingen te kunnen doen over effectiviteit wordt van sociale professionals vereist dat zij een gestructureerde en consistente werkwijze hanteren, volgens gestandaardiseerde protocollen (Otto et al, 2009).…”

Section: T H E O R E T I S C H O N D E R B O U W E N Va N S O C I a Lunclassified

Een Program Theory benadering voor het theoretisch onderbouwen van sociale interventies: een casestudie van vijf Nederlandse maatjesprojecten

Tier¹,

Hermans²,

Potting³

2016

JSI

View full text Add to dashboard Cite

“…The deployment of RCTs and related experimental designs in policy evaluation has proved controversial in policy areas beyond energy-the adoption of RCTs within UK criminal justice policy-making in the late 1990s led to a small revolution within the UK evaluation community embodied by Pawson and Tilley's publication of Realistic Evaluation which was as much a manifesto for theory-driven evaluation as it was a rail against the application of RCTs (Pawson and Tilley 1997). Despite recent attempts to bring realist evaluation approaches together with RCT and experimental designs (Bonell et al 2012), the controversy continues (Marchal et al 2013) and the compatibility of RCTs with policy-making should not be taken as a given .…”

Section: Introductionmentioning

confidence: 99%

Evaluating energy efficiency policy: understanding the ‘energy policy epistemology’ may explain the lack of demand for randomised controlled trials

Cooper¹

2018

Energy Efficiency

View full text Add to dashboard Cite

Vine et al. 's (2014) call for more randomised controlled trials (RCTs) in government-funded energy efficiency policy evaluation practice raises timely questions about what constitutes effective designs for evaluating and informing energy policy. Their implicit hypothesis that policy organisations share the same epistemic perspective as they do, and that the reason there are few RCTs are due to a set of barriers to be overcome is examined in relation to the UK government Department of Energy and Climate Change. Drawing on the author's experience of working in the ministry, the claim that barriers are a reason for preventing RCT use is discounted. An alternative explanation is presented, framed around the idea of an 'energy policy epistemology' that legitimately places certain specific knowledge demands and ways of knowing on research and evaluation designs. Through examination of a specific set of research and evaluation outputs related to the UK energy efficiency policy called the 'Green Deal', aspects of the proposed 'energy policy epistemology' are elucidated to explain the lack of demand for RCT designs. Final consideration is given to what kinds of designs are more likely to gain support in this context that might also deliver many of the benefits attributed to RCTs with longitudinal panels being one important example.

show abstract

Realist RCTs of complex interventions – An oxymoron

Cited by 133 publications

References 15 publications

The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance

The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance

Een Program Theory benadering voor het theoretisch onderbouwen van sociale interventies: een casestudie van vijf Nederlandse maatjesprojecten

Evaluating energy efficiency policy: understanding the ‘energy policy epistemology’ may explain the lack of demand for randomised controlled trials

Contact Info

Product

Resources

About