Evaluating the Impact of Meningococcal Vaccines With Synthetic Controls

Abstract

Invasive meningococcal disease (IMD) has a low and unpredictable incidence, presenting challenges for real-world evaluations of meningococcal vaccines. Traditionally, meningococcal vaccine impact is evaluated by predicting counterfactuals from pre-immunization IMD incidences, possibly controlling for IMD in unvaccinated age groups, but the selection of controls can influence results. We retrospectively applied a synthetic control (SC) method, previously used for pneumococcal disease, to two infant immunization programs against serogroups B and C IMD in England and Brazil. Time series of infectious/non-infectious diseases in infants and IMD cases in older unvaccinated age groups were used as candidate controls, automatically combined in a SC through Bayesian variable selection. SC closely predicted IMD in absence of vaccination, adjusting for non-trivial changes in IMD incidence. Vaccine impact estimates were in line with previous assessments. IMD cases in unvaccinated age groups were the most frequent SC-selected controls. Similar results were obtained when excluding IMD from control sets and using other diseases only, particularly respiratory diseases and measles. Using non-IMD controls may be important where there are herd immunity effects. SC is a robust and flexible method that addresses uncertainty introduced when equally plausible controls exhibit different post-immunization behaviors, allowing objective between-countries comparisons of IMD programs.

Quasi-experimental methods for pharmacoepidemiology: difference-in-differences and synthetic control methods with case studies for vaccine evaluation

“…Quantitative placebo tests can also allow the reporting of causal uncertainty alongside statistical uncertainty. 3 Investigators can improve generalizability by using multiple populations (as in the meningococcal vaccine case study 39 ), different methodological approaches or models (as in the COVID-19 vaccine case study 46 ), or multiple units and exposure points (as in the BCG vaccine case study 44 ). The latter, however, targets an estimand that averages potentially heterogeneous treatment effects, which may or may not have greater generalizability—depending on the setting—and can be statistically inefficient.…”

“…Prunas et al analyzed the impact of early childhood meningococcal vaccination programs in Brazil and England in the 2010s (see Table 1 ). 39 Real-world estimates of vaccine effectiveness for hard-to-predict and highly variable diseases like invasive meningococcal disease are challenging, making quasi-experiments particularly useful. Using SCM on the logarithmic scale with Bayesian time-series modeling, the authors compared meningococcal disease incidence after the program roll-out with a synthetic control constructed from time series of other (nontargeted) diseases and the targeted disease in older age groups who were ineligible for vaccination.…”

“…The authors used placebo tests and sensitivity analyses that dropped the older age groups as candidate control series, finding robust results that suggest these biases were minimal in this setting. 39 …”