Comparison of statistical methods for integrating real‐world evidence in a rare events meta‐analysis of randomized controlled trials

Abstract

Rare events meta‐analyses of randomized controlled trials (RCTs) are often underpowered because the outcomes are infrequent. Real‐world evidence (RWE) from non‐randomized studies may provide valuable complementary evidence about the effects of rare events, and there is growing interest in including such evidence in the decision‐making process. Several methods for combining RCTs and RWE studies have been proposed, but the comparative performance of these methods is not well understood. We describe a simulation study that aims to evaluate an array of alternative Bayesian methods for including RWE in rare events meta‐analysis of RCTs: the naïve data synthesis, the design‐adjusted synthesis, the use of RWE as prior information, the three‐level hierarchical models, and the bias‐corrected meta‐analysis model. The percentage bias, root‐mean‐square‐error, mean 95% credible interval width, coverage probability, and power are used to measure performance. The various methods are illustrated using a systematic review to evaluate the risk of diabetic ketoacidosis among patients using sodium/glucose co‐transporter 2 inhibitors as compared with active‐comparators. Our simulations show that the bias‐corrected meta‐analysis model is comparable to or better than the other methods in terms of all evaluated performance measures and simulation scenarios. Our results also demonstrate that data solely from RCTs may not be sufficiently reliable for assessing the effects of rare events. In summary, the inclusion of RWE could increase the certainty and comprehensiveness of the body of evidence of rare events from RCTs, and the bias‐corrected meta‐analysis model may be preferable.