Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses

Abstract

Evidence on how much medical interventions work may change over time. It is important to determine what fluctuations in the treatment effect reported by randomized trials and their metaanalyses may be expected and whether extreme fluctuations signal future major changes. We applied recursive cumulative metaanalysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 interventions in two medical fields (pregnancy͞perinatal medicine, n ‫؍‬ 45 interventions; myocardial infarction, n ‫؍‬ 15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes greater than 2.5 standard deviations of the expected. With 500 accumulated patients, the pooled odds ratio may change by 0.6-to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74-and 1.35-fold for pregnancy͞ perinatal medicine and between 0.83-and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of preeclampsia), where recent mega-trials have contradicted prior metaanalyses, as well as in four other examples where early large treatment effects were dissipated when more data appeared. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their metaanalyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may sometimes signal further major changes in the future. mega-trials ͉ heterogeneity ͉ randomized trials R andomized trials and metaanalyses are often considered primary means for assessing the efficacy of medical interventions (1-4). However, clinical evidence evolves over time: new trials continue to be performed, replacing, updating, and supplementing the knowledge obtained from earlier ones. Heterogeneity (i.e., variability) among trial results is unavoidable. New trials may strengthen our prior beliefs about the magnitude of a treatment effect (i.e., how much a treatment works), or, in some cases, they may alter these beliefs or invalidate them. Several recent examples (5-8) have been encountered where large randomized trials reached entirely different conclusions when compared with metaanalyses of earlier trials of small or even large sample size on the same question. For example, recent large trials seemed to invalidate our prior beliefs about the efficacy of treatments such as magnesium salts and nitrates for reducing overall mortality in acute myocardial infarction (9) or aspirin (10) and calcium supplementation (11) for the prevention of preeclampsia during pregnancy. Prior expectations of 30-60% reductions in mortality and preeclampsia, respectively, based on early trials, were not confirmed.Important questions arise: Could we have anticipated th...

Role of the Fcγ receptor IIA polymorphism in the antiphospholipid syndrome: An international meta‐analysis

“…Finally, we performed cumulative meta-analysis and recursive cumulative meta-analysis (27,28) to evaluate whether the combined OR changed over time as more data were accumulated for each comparison and contrast. Inverted funnel plots (29) were examined as diagnostics for heterogeneity related to the sample size of each study.…”

Conceptualizing changes in behavior in intervention research: The range of possible changes model.

“…This research is critical to the understanding of intervention change in this area of science. Indeed, often times initial studies finding positive effects of interventions are subsequently contradicted by further well-conducted research on these same interventions; subsequent research often finds no evidence for positive intervention effects, or even evidence of significantly negative intervention effects (e.g., Ioannidis, 2005;Ioannidis & Lau, 2001;Ioannidis & Trikalinos, 2005;Trikalinos et al, 2004).It is likely that if replication studies in the published psychological intervention literatures were as common as they are in published literatures of the general medical sciences, these same inconsistencies may be observed. Thus, all well-conducted intervention studies, regardless of their RPC Model classification, speak to the wide variability of change.…”

“…This research is critical to the understanding of intervention change in this area of science. Indeed, often times initial studies finding positive effects of interventions are subsequently contradicted by further well-conducted research on these same interventions; subsequent research often finds no evidence for positive intervention effects, or even evidence of significantly negative intervention effects (e.g., Ioannidis, 2005;Ioannidis & Lau, 2001;Ioannidis & Trikalinos, 2005;Trikalinos et al, 2004).…”

The association of P‐glycoprotein with response to chemotherapy and clinical outcome in patients with osteosarcoma

“…We also used appropriate bias diagnostics to examine whether there was evidence that the results differed in small studies compared with large studies 22 or whether results were changing gradually over time with the publication of more recent studies. 23 Analyses were conducted in using SPSS (version 10.0; SPSS, Inc., Chicago, IL), Meta-Analyst (Joseph Lau, Boston, MA) and Meta-Test (Joseph Lau) software packages. P values are two-tailed.…”