Assessing the Gold Standard — Lessons from the History of RCTs

Bothwell, Laura E.; Greene, Jeremy A.; Podolsky, Scott H.; Jones, David S.

doi:10.1056/nejmms1604593

Cited by 481 publications

(389 citation statements)

References 45 publications

Supporting

Mentioning

380

Contrasting

Unclassified

Order By: Relevance

“…84 Moreover, there is growing interest in the use of observational data, including big data and real-world data, to make causal inferences about the efficacy of novel treatments. 85,86 One justification for this interest is that it may not be feasible or ethical to conduct RCTs for highly promising medical practices.…”

Section: Discussionmentioning

confidence: 99%

Most medical practices are not parachutes: a citation analysis of practices felt by biomedical authors to be analogous to parachutes

Hayes¹,

Kaestner

Mailankody

et al. 2018

CMAJ Open

View full text Add to dashboard Cite

Background:In a 2003 paper in BMJ, the authors made the tongue-in-cheek observation that there are no randomized controlled trials (RCTs) of parachutes. This paper has been widely read, cited and used to argue that RCTs are impractical or unnecessary for some medical practices. We performed a study to identify and evaluate claims that a medical practice is akin to a parachute. Methods:Using Google Scholar, we identified all citations to the 2003 paper. We searched for claims that a specific practice was akin to a parachute. For each practice, we identified the desired outcome of the practice, and searched Google Scholar and ClinicalTrials.gov for RCTs that were conducted, ongoing, halted, planned or unpublished.Results: Of 822 articles citing the original paper, 35 (4.1%) argued that a medical practice was akin to a parachute. Eighteen of the 35 (51%) concerned mortality or live birth, and 17 (49%) concerned a lesser outcome. For 22 practices (63%), we identified 1 or more RCTs: in 6 cases (27%), the trials showed a statistically significant benefit of the practice; in 5 (23%), the trials rejected the practice; in 5 (23%), the trials had mixed results; in 2 (9%), the trials were halted; and in 4 (18%), the trials were ongoing. Effect size was calculated for 5 of the 6 practices for which RCTs gave positive results, and the absolute risk reduction ranged from 11% to 30.8%, corresponding to a number needed to treat of 3-9.Interpretation: Although there is widespread interest regarding the BMJ paper arguing that randomized trials are not necessary for practices of clear benefit, there are few analogies in medicine. Most parachute analogies in medicine are inappropriate, incorrect or misused. Abstract Research OPEN Research E32CMAJ OPEN, 6(1)

show abstract

Section: Discussionmentioning

confidence: 99%

Most medical practices are not parachutes: a citation analysis of practices felt by biomedical authors to be analogous to parachutes

Hayes¹,

Kaestner

Mailankody

et al. 2018

CMAJ Open

View full text Add to dashboard Cite

show abstract

“…Pre-specified endpoints, controls, randomization and blinding cannot be discarded without sacrificing actionable clinical information 5 .…”

Section: No Free Lunchmentioning

confidence: 99%

Show drugs work before selling them

Sipp

McCabe

Rasko

2017

Nature

View full text Add to dashboard Cite

The field of quantum computing will soon achieve a historic milestone -quantum supremacy. It is still unknown whether application-related algorithms will be able to deliver big increases in speed using the sorts of processors that will soon be available. But when quantum hardware becomes sufficiently powerful, it will become possible to test this and develop new types of algorithms.Over the next decade, academia, industry and national labs should work together to develop quantum simulation and quantum machine-learning algorithms. We plan to support such research by offering access to Google's quantum processors through cloud services to others that lack the necessary capital, expertise or infrastructure. ■ Masoud Mohseni is senior research scientist at

show abstract

“…Much of the power of this argument comes from an implicit idea that since RCTs are used in medical research, and since medical research has clearly provided many benefits to humanity, the use of RCTs must therefore by extension be good in other domains. However, even in the medical research, the use of RCTs is far from either uncontested or seen as a 'gold standard', as Bothwell et al (2016) elegantly demonstrate. In medical research, RCTs are seen as just one kind of evidence-mainly in pharmaceutical settingsand other evidence (from qualitative case studies to observational epidemiology) are more commonly utilised in practice.…”

Section: Introductionmentioning

confidence: 99%

Evaluating energy efficiency policy: understanding the ‘energy policy epistemology’ may explain the lack of demand for randomised controlled trials

Cooper¹

2018

Energy Efficiency

View full text Add to dashboard Cite

Vine et al. 's (2014) call for more randomised controlled trials (RCTs) in government-funded energy efficiency policy evaluation practice raises timely questions about what constitutes effective designs for evaluating and informing energy policy. Their implicit hypothesis that policy organisations share the same epistemic perspective as they do, and that the reason there are few RCTs are due to a set of barriers to be overcome is examined in relation to the UK government Department of Energy and Climate Change. Drawing on the author's experience of working in the ministry, the claim that barriers are a reason for preventing RCT use is discounted. An alternative explanation is presented, framed around the idea of an 'energy policy epistemology' that legitimately places certain specific knowledge demands and ways of knowing on research and evaluation designs. Through examination of a specific set of research and evaluation outputs related to the UK energy efficiency policy called the 'Green Deal', aspects of the proposed 'energy policy epistemology' are elucidated to explain the lack of demand for RCT designs. Final consideration is given to what kinds of designs are more likely to gain support in this context that might also deliver many of the benefits attributed to RCTs with longitudinal panels being one important example.

show abstract

Assessing the Gold Standard — Lessons from the History of RCTs

Cited by 481 publications

References 45 publications

Most medical practices are not parachutes: a citation analysis of practices felt by biomedical authors to be analogous to parachutes

Most medical practices are not parachutes: a citation analysis of practices felt by biomedical authors to be analogous to parachutes

Show drugs work before selling them

Evaluating energy efficiency policy: understanding the ‘energy policy epistemology’ may explain the lack of demand for randomised controlled trials

Contact Info

Product

Resources

About