Theoretical Framework for Retrospective Studies of the Effectiveness of SARS-CoV-2 Vaccines

Abstract

Observational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. Such studies are now underway amid the ongoing rollout of SARS-CoV-2 vaccines globally. While traditional case-control and test-negative design studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here we review the theoretical basis for estimation of vaccine direct effects under traditional case-control and test-negative design frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs.Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, non-specific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The traditional case-control design may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The test-negative design reduces but may not eliminate this confounding, for instance if individuals who receive vaccination seek care or testing for less-severe illness. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources.We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages.

Effectiveness of the CoronaVac vaccine in older adults during a gamma variant associated epidemic of covid-19 in Brazil: test negative case-control study

“…Furthermore, we estimated the effectiveness of a single dose 0-13 days after the first dose, when the vaccine has shown no or limited effectiveness 53334. An association during this period might serve as an indicator of unmeasured confounding in the effectiveness estimate 3536. We also expanded our bias indicator by evaluating the 0-13 days after the first dose as 0-6 days and 7-13 days 36.…”

“…An association during this period might serve as an indicator of unmeasured confounding in the effectiveness estimate 3536. We also expanded our bias indicator by evaluating the 0-13 days after the first dose as 0-6 days and 7-13 days 36. The reference group for vaccination status was those who had not received a first vaccine dose before the date of sample collection.…”

Effectiveness of COVID-19 vaccines in preventing SARS-CoV-2 infection and hospitalisation, Navarre, Spain, January to April 2021

“…Compared with the test-negative and specific-cohort designs, the cohort of close contacts has advantages for bias control, since participants had a similar risk exposure regardless their vaccination status, and provided a good representativeness of the general population [20]. Only close contacts tested for COVID-19 were included in this study.…”

“…As close contacts of COVID-19 cases have had a known risk exposure, the comparison of COVID-19 incidence between vaccinated and unvaccinated close contacts is an ideal design to assess the COVID-19 VE. Characteristics of the individuals included in the analysis according to their vaccination status, Navarre, Spain, January-April 2021 (n =20,961)…”

Vaccine Effectiveness Studies in the Field

“…In test-negative design studies, the confounding effects of health care–seeking behavior is reduced; however, other biases will occur. 7 In comparing the effectiveness of vaccines against virus variants, some biases may be the same for each variant. In the PHE study, the two vaccines were used in different ways over time and were available in different health care settings and in different age groups at different times, thus making valid comparison difficult.…”