The test-negative design for estimating influenza vaccine effectiveness

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

“…As a corollary, the number of potential biases identified for a particular design is not an indicator of the design's desirability, as these biases may be of trivial magnitude in particular cases. Some work has been done, but more work is needed, to estimate the likely magnitude of these biases (26,27,41,(61)(62)(63). Such work is important because of the high reliance on such studies to provide of the evidence base used to set vaccine policy.. Failure to account for such biases has led to large errors in the estimates of vaccine effects in observational studies, and that these errors may be amplified as they gain authority and apparent precision through the process of systematic review and meta-analysis.…”

“…That flawed evidence base had suggested that 1 death could be saved by vaccinating 150-300 elderly -when in fact seniors respond poorly to influenza vaccine. Once the cohort study designs had been improved and bias detection strategies incorporated (17,18,62) earlier reports of astonishing mortality savings were replaced with the insight that these studies suffered from confounding that led to dramatic VE overestimation. This finding was reinforced by evidence from a trend study of modest effects at the population level (15).…”

Estudio cycEVA: casos y controles para la estimación de la efectividad de la vacuna antigripal en España, 2008-2013

“…Con respecto a los sesgos metodológicos del diseño test-negativo de casos y controles empleado cabe destacar que, aunque actualmente es el diseño más utilizado en los estudios observacionales sobre la efectividad de la vacuna antigripal, se siguen describiendo ciertos sesgos asociados con su uso 4,[10][11][12][13][14] . Aunque se asume que los controles negativos y los casos presentan la misma conducta de frecuentación médica (health seeking behaviour), se sigue cuestionando si los controles reflejan adecuadamente la cobertura vacunal de la población que da origen a los casos.…”

“…Los países participantes en la red I-MOVE consensuaron un protocolo común 7 que incluye una toma sistemática de muestra respiratoria a los pacientes participantes, registro de información sobre posibles factores de confusión y el uso de la definición de caso de gripe de la UE 8 . A partir de la fase piloto 2008-09 se utilizó el diseño test-negativo de casos [9][10][11][12][13] , estimán-dose la efectividad vacunal (EV) antigripal frente a la infección de gripe confirmada por laboratorio. Los casos son los sujetos que consultan al médico centinela por síndrome gripal y con resultado virológico positivo para virus gripales.…”

Reply: Letter to the Editor, Cochrane Rearranged

“…The main theoretical problem of this design is the same as in all observational studies: the exposure risk may not be the same in vaccinated and unvaccinated persons [6] potentially leading to a large variation of the effectiveness estimates between places and seasons. Single study results from a limited number of seasons, like the 27% and 9% point estimates mentioned by Osterholm et al, should therefore be interpreted with caution, and cannot easily be compared with our averages and ranges based on 40 years of observations across numerous studies.…”