Meta-analysis of rare adverse events in randomized clinical trials: Bayesian and frequentist methods

Abstract

Background/aims: Regulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events. Methods: In this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes. Results: Our simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals. Conclusion: In the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.

Physical Activity and Cognition in Sedentary Older Adults: A Systematic Review and Meta-Analysis

“…Our study has the following strengths: 1) we conducted a comprehensive literature search using nine medical databases, and the study selection and quality assessments were performed independently by two researchers, which ensured strict quality control during the study process, including data collection; 2) all studies in our analysis were RCTs, which is the most common type of interventional study and has certain advantages over other types of studies [ 45 ]; 3) we measured the subjects’ general characteristics at baseline, and the included population was homogenous. Through subgroup analysis, the baseline status of each group was relatively consistent, the comparability was good, bias was low, and the external implementation of the results was strong [ 46 ]; 4) despite the heterogeneity observed in the study, the consistent results of the sensitivity analysis indicate that our findings are reliable and robust.…”

Bayesian Methods for Meta-Analyses of Binary Outcomes: Implementations, Examples, and Impact of Priors

“…We have summarized several possible solutions to handle this problem in Figure 1. If the Bayesian meta-analysis still fails to produce valid results after considering these solutions, researchers may refer to several papers that especially focused on the cases of few studies [38,[64][65][66][67] and rare events [68][69][70][71][72][73][74][75] for alternative approaches. Moreover, if individual participant data are available, incorporating them into the metaanalysis might help to improve the estimation of treatment effects [13,[76][77][78].…”

“…Instead, we aimed at providing some practical guidelines for researchers who are interested in implementing Bayesian meta-analyses. One may refer to other papers that especially aimed at comparing these two types of methods to better understand the pros and cons of frequentist and Bayesian meta-analyses [74,[79][80][81][82][83][84].…”

Restoring invisible and abandoned trials of gabapentin for neuropathic pain: a clinical and methodological investigation

“…For the second stage, we will consider various meta-analytic methods to fit rare binary outcomes. 55 We will fit traditional frequentist methods including Peto and Mantel-Haenszel and Bayesian hierarchical meta-analytic methods that incorporate between-study heterogeneity with random effects. This method is preferable to a frequentist approach because studies with ‘zero’ cells are not a problem for the Peto, Mantel-Haenszel and Bayesian methods.…”