Evaluation of post-introduction COVID-19 vaccine effectiveness: Summary of interim guidance of the World Health Organization

Abstract

Phase 3 randomized-controlled trials have provided promising results of COVID-19 vaccine efficacy, ranging from 50-95% against symptomatic disease as the primary endpoints, resulting in emergency use authorization/listing for several vaccines. However, given the short duration of follow-up during the clinical trials, strict eligibility criteria, emerging variants of concern, and the changing epidemiology of the pandemic, many questions still remain unanswered regarding vaccine performance. Post-introduction vaccine effectiveness evaluations can help to understand the effect on reducing infection and disease when used in real-world conditions. They can also address important questions that were either not studied or were incompletely studied in the trials and that will inform evolving vaccine policy, including assessment of the duration of effectiveness; effectiveness in key subpopulations, such as the very old or immunocompromised; against severe disease and death due to COVID-19; against emerging SARS-CoV-2 variants of concern, and with different vaccination schedules, such as number of doses and varying dosing intervals. WHO convened an expert panel to develop interim best practice guidance for COVID-19 vaccine effectiveness evaluations. We present a summary of the interim guidance, including discussion of different study designs, priority outcomes to evaluate, potential biases, existing surveillance platforms that can be used, and recommendations for reporting results.

Effectiveness of the CoronaVac vaccine in older adults during a gamma variant associated epidemic of covid-19 in Brazil: test negative case-control study

“…A preliminary aggregated analysis using weekly times series of covid-19 deaths in Brazil found a relative decrease in mortality among those aged ≥70 years compared with all ages after vaccination with CoronaVac and ChAdOx1 nCov-19,45 suggesting a discernible impact of vaccination on mortality at the population level. Additional investigation is required to determine the duration of protection conferred by CoronaVac 71923…”

“…This study has several strengths, which include the large sample size and geospatial coverage, comprising the State of São Paulo with 46 million inhabitants distributed across 645 municipalities. We implemented a prespecified publicly available protocol, which is in accordance with the recent WHO guideline for evaluation of covid-19 vaccine effectiveness 23. Using a test negative design, we have dealt with biases that affect observational studies on vaccine effectiveness, such as health seeking behaviour and access.…”

“…We conducted a matched test negative case-control study to estimate the effectiveness of CoronaVac in reducing the odds of reverse transcription polymerase chain reaction (RT-PCR) confirmed symptomatic covid-19 in adults aged ≥70 years from São Paulo state from 17 January 2021 (the start of covid-19 vaccination) to 29 April 2021. Test negative design studies have provided estimates of vaccine effectiveness in concordance with those obtained from randomised controlled trials,2728 and such studies have been used extensively to evaluate vaccines against respiratory infections,29 including covid-19 82330. We chose the test negative design because of the feasibility of accessing information on people who were tested for SARS-CoV-2 through the São Paulo state surveillance systems and because of the opportunity to control for potential biases, such as healthcare seeking behaviour and access to testing 23.…”

“…Test negative design studies have provided estimates of vaccine effectiveness in concordance with those obtained from randomised controlled trials,2728 and such studies have been used extensively to evaluate vaccines against respiratory infections,29 including covid-19 82330. We chose the test negative design because of the feasibility of accessing information on people who were tested for SARS-CoV-2 through the São Paulo state surveillance systems and because of the opportunity to control for potential biases, such as healthcare seeking behaviour and access to testing 23. The study population was adults aged ≥70 years who had a residential address in São Paulo state, underwent SARS-CoV-2 RT-PCR testing during the study period, and had complete and consistent information between data sources on age, sex, residence, and on vaccination and testing status and dates.…”

“…One case was matched with to up to five test negative controls according to RT-PCR sample collection date (±3 days), age category (five year age bands, eg, 70-74, 75-79 years), sex, municipality of residence, self-reported race (defined as brown, black, yellow, white, or indigenous),32 and previous symptomatic events that were reported to the surveillance systems31 between 1 February 2020 and 16 January 2021, as a proxy for previous SARS-CoV-2 infection. Matching factors were chosen from variables that were associated with vaccination coverage or timing, and with risk of SARS-CoV-2 infection or healthcare access (see protocol in supplementary file) 23. After identification of each case, we randomly chose up to five controls in an unbalanced design from the set of all eligible matching controls, allowing for replacement of controls between cases (main analysis).…”