Estimation of vaccination coverage from electronic healthcare records; methods performance evaluation – A contribution of the ADVANCE-project

Abstract

IntroductionThe Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines, using existing electronic healthcare record (eHR) databases in Europe.Part of the data in such sources is missing due to incomplete follow-up hampering the accurate estimation of vaccination coverage. We compared different methods for coverage estimation from eHR databases; naïve period prevalence, complete case period prevalence, period prevalence adjusted for follow-up time, Kaplan-Meier (KM) analysis and (adjusted) inverse probability weighing (IPW).MethodsWe created simulation scenarios with different proportions of completeness of follow-up. Both completeness independent and dependent from vaccination date and status were considered. The root mean squared error (RMSE) and relative difference between the estimated and true coverage were used to assess the performance of the different methods for each of the scenarios. We included data examples on the vaccination coverage of human papilloma virus and pertussis component containing vaccines from the Spanish BIFAP database.ResultsUnder completeness independent from vaccination date or status, several methods provided estimates with bias close to zero. However, when dependence between completeness of follow-up and vaccination date or status was present, all methods generated biased estimates. The IPW/CDF methods were generally the least biased. Preference for a specific method should be based on the type of censoring and type of dependence between completeness of follow-up and vaccination. Additional insights into these aspects, might be gained by applying several methods.

Methodological frontiers in vaccine safety: qualifying available evidence for rare events, use of distributed data networks to monitor vaccine safety issues, and monitoring the safety of pregnancy interventions

“…There have also been bipartite collaborations such as Denmark and Sweden for human papillomavirus (HPV) vaccine and autoimmune conditions 32 33. Most recently, the ADVANCE project created a system for monitoring vaccine benefits and risks and is now continuing as the VAC4EU 20 34–39. Most of these studies have involved data sharing to a hub and while some have involved a single methodology, others have allowed different study designs.…”

Base de Datos para la Investigación Farmacoepidemiológica en Atención Primaria (BIFAP): A data resource for pharmacoepidemiology in Spain

“…Furthermore, comparisons of incidence/prevalence of clinical events and of exposure to medicines in BIFAP with external figures has been evaluated in benchmarking processes through international multi-database projects 16,20,21 and with population-based registries. 18,22 Regarding medicines exposure data non-standardized information registered in the EMRs on vaccinations has been validated by manual review of patient profiles. 23 In addition, an algorithm to identify pregnancies in BIFAP has been developed to allow studying the use and effects of drugs during pregnancy.…”

“…18,22 Regarding medicines exposure data non-standardized information registered in the EMRs on vaccinations has been validated by manual review of patient profiles. 23 In addition, an algorithm to identify pregnancies in BIFAP has been developed to allow studying the use and effects of drugs during pregnancy. 24 BIFAP is fully integrated in the AEMPS roles and activities as a useful tool for generating scientific evidence on medicines related issues, helping regulatory decision making.…”

“…Vaccines (J07) Pertussis vaccine adverse events 48 Coverage 22,49,50 Anti-inflammatory drugs (M01) Myocardial infarction 51 Colorectal cancer 52 Gastrointestinal bleeding 53 Ischemic stroke 54…”

“…Manual review of free text has also been performed as part of the case definition procedure within drug‐event association studies (see Table 2). Furthermore, comparisons of incidence/prevalence of clinical events and of exposure to medicines in BIFAP with external figures has been evaluated in benchmarking processes through international multi‐database projects 16,20,21 and with population‐based registries 18,22 …”