On the treatment effect heterogeneity of antidepressants in major depression: A Bayesian meta-analysis and simulation study

Lisinski

et al. 2021

Acta Psychiatr Scand

Objective Antidepressants outperform placebo with an effect size of around 0.30. It has been suggested that effect sizes as high as 0.875 are necessary for a minimal clinically important difference. Whether such effect sizes are achievable in placebo‐controlled trials is unknown. Therefore, we aimed to assess what effect sizes are theoretically achievable in placebo‐controlled trials of antidepressants. Methods Patient‐level analyses comparing Hamilton Depression Rating Scale (HDRS‐17) outcomes for simulated antidepressant therapies to placebo‐treated participants (n = 2201) from clinical trials of selective serotonin reuptake inhibitors. Results An optimally effective antidepressant, where all treated participants achieve HDRS‐17 scores comparable to those displayed by healthy volunteers (remission‐type model), had a maximum effect size of 1.75, with a mean difference of 11.6 points on the HDRS‐17. In simulations where patients received an additional 50% symptom reduction over that obtained with placebo (improvement‐type model), the maximum effect size was 1.08 with a mean HDRS‐17 difference of 7.2. When adjusting for normal rates of treatment discontinuation, maximum effect sizes were 1.10 (remission‐type model) and 0.76 (improvement‐type model) with HDRS‐17 mean differences of 8.8 and 5.6, respectively. Conclusions Three methodological issues (i) a large and variable placebo response, (ii) a high rate of dropout and (iii) HDRS‐17‐ratings significantly larger than zero in healthy volunteers, reduce the degree of treatment‐placebo separation achievable in depression trials. Assuming that those who discontinue treatment have only partial response, even a highly effective antidepressant would have difficulties surpassing such effect size cut‐offs as have been suggested to signify a minimal clinically important difference.

Section: Discussionsupporting

confidence: 82%

mentioning

confidence: 92%

Determining maximal achievable effect sizes of antidepressant therapies in placebo‐controlled trials

Lisinski

et al. 2021

Acta Psychiatr Scand

“…When slope estimates are closer to 0 or PLOS BIOLOGY nonlinearities are present in the mean-variance relationship, other metrics of variability such as log variability ratio (lnVR) or an approach that directly estimates the strength of association between log mean and log variance (i.e., an arm-based meta-analysis [69]) based on log SD may be more appropriate (for an example of an arm-based approach, see S9 Table and S4 Fig for galaxy plot of lnRR on lnSD). We advise that future analyses of heterogeneity pick the most appropriate statistic and model of variability based on the mean-variance relationships present in their dataset [71]. In addition to assessing the effects of treatments on variance, we further quantified differences in mean infarct volume by calculating the lnRR of the mean for each control/experimental group within a study (lnRR) and its associated sampling variance (s 2 lnRR ).…”

Section: Calculating Effect Sizesmentioning

confidence: 99%

Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research

et al. 2021

The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as “heterogenization” of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research.

“…Several analyses of variability based on study-level data, however, found no evidence of greater variability in the active treatment arm, and thus concluded that treatment effects are likely to be relatively constant, which suggests the scope for personalisation of treatment is limited (7)(8)(9)(10)(11)(12)(13). These findings were surprising given the widespread clinical belief that patients differ substantially in their response to medication.…”

Section: Introductionmentioning

confidence: 99%

“…Meta-analysis of variability has recently become an area of intense interest in psychiatric research. The technique has been applied to understand variability of brain structure (1), and subsequently brain function (2)(3)(4)(5), immune function (6), and more recently to investigate variability in randomised controlled trials (RCTs) of therapeutic interventions thereby potentially providing an estimate of treatment effect heterogeneity (7)(8)(9)(10)(11)(12)(13).…”

Section: Introductionmentioning

confidence: 99%

Reappraising Treatment Effect Heterogeneity in Schizophrenia: A Meta-analysis

McCutcheon

Pillinger

Efthimiou

et al. 2021

Preprint

ObjectiveDetermining whether individual patients differ in response to treatment (‘treatment effect heterogeneity’) is important as it is a prerequisite to developing personalised treatment approaches. Previous variability meta-analyses of response to antipsychotics in schizophrenia found no evidence for treatment effect heterogeneity. Conversely, individual patient data meta-analyses suggest treatment effect heterogeneity does exist. In the current paper we combine individual patient data with study level data to resolve this apparent contradiction and quantitively characterise antipsychotic treatment effect heterogeneity in schizophrenia.MethodIndividual patient data (IPD) was obtained from the Yale University Open Data Access (YODA) project. Clinical trials were identified in EMBASE, PsycInfo, and PubMed. Treatment effect heterogeneity was estimated from variability ratios derived from study-level data from 66 RCTs of antipsychotics in schizophrenia (N=17,202). This estimation required a correlation coefficient (ρ) between placebo response and treatment effects to be estimated. We estimated this from both study level estimates of the 66 trials, and individual patient data (N=560).ResultsBoth individual patient (ρ=-0.32, p=0.002) and study level (ρ=-0.38, p<0.001) analyses yielded a negative correlation between placebo response and treatment effect. Using these estimates we found evidence of clinically significant treatment effect heterogeneity for total symptoms (our most conservative estimate was SD = 13.5 Positive and Negative Syndrome Scale (PANSS) points). Mean treatment effects were 8.6 points which, given the estimated SD, suggests the top quartile of patients experienced beneficial treatment effects of at least 17.7 PANSS points, while the bottom quartile received no benefit as compared to placebo.ConclusionsWe found evidence of clinically meaningful treatment effect heterogeneity for antipsychotic treatment of schizophrenia. This suggests efforts to personalise treatment have potential for success, and demonstrates that variability meta-analyses of RCTs need to account for relationships between placebo response and treatment effects.