Protocols for meta-analysis of intervention safety seldom specified methods to deal with rare events

Abstract

Objectives: Meta-analyses of rare events often generate unstable results, and selective reporting of the results may mislead the health care decision. Developing a synthesis plan for rare events in protocol may help to formulate the reporting. We aim to investigate whether existing protocols specified methods to deal with rare events. Study Design and Setting: Protocols (not including Cochrane protocols) for systematic reviews of health care interventions focused on the safety registered in PROSPERO were included. The proportion of protocols that specified methods to deal with rare events and the detailed methods were summarized. We compared the difference of proportions (PD) across different settings. Results: We identified 1,004 eligible protocols, of which, 119 (11.85%, 95% confidence interval (CI): 9.92%, 14.01%) specified methods to deal with rare events. The three most commonly planned methods were the ManteleHaenszel, Peto's odds ratio, and continuity correction. Protocols planned a quantitative analysis (PD 5 0.07, 95% CI: 0.02, 0.12; P 5 0.004) and listed safety as a primary outcome (PD 5 0.07, 95% CI: 0.01, 0.12; P 5 0.018) were more likely to specify methods to deal with rare events. Conclusion: Protocols for systematic reviews of intervention safety seldom specified methods to deal with rare events. Future systematic reviewers should provide a detailed and rigorous synthesis plan for rare events in their protocols.

A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies

“…However, it is highly recommended for authors to develop a synthesis plan in advance to avoid selective reporting, including for meta-analysis that may involve zero-events [35,36]. One possible and practical solution is that researchers consider all six possible situations a priori and pre-define one or more available methods for each situation in their protocols.…”

Random‐effects meta‐analysis models for the odds ratio in the case of rare events under different data‐generating models: A simulation study

“…Although this model has been recommended for rare events meta‐analysis (Stijnen et al., 2010), we are not aware of any comprehensive simulation study assessing its performance in this setting. The present simulation study aims at filling this research gap by comparing the performance of the hypergeometric‐normal GLMM to its most significant competitors, including the beta‐binomial model which we describe below, the binomial‐normal GLMM, as well as the inverse variance model and the MH method, which are both frequently used in rare events meta‐analysis (Zhou et al., 2020).…”

Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

“…The proportions were also compared in terms of whether an a priori protocol for handling zero events was developed or not. The proportion difference (PD) was used to measure the difference as a valid method for measuring the effect even in the presence of zero counts [ 26 , 28 ]. Fisher’s exact test was used for sensitivity analysis when events were rare (i.e., less than 5).…”