Pragmatic considerations for negative control outcome studies to guide non‐randomized comparative analyses: A narrative review

Abstract

Purpose: This narrative review describes the application of negative control outcome (NCO) methods to assess potential bias due to unmeasured or mismeasured confounders in non-randomized comparisons of drug effectiveness and safety. An NCO is assumed to have no causal relationship with a treatment under study while subject to the same confounding structure as the treatment and outcome of interest; an association between treatment and NCO then reflects the potential for uncontrolled confounding between treatment and outcome. Methods:We focus on two recently completed NCO studies that assessed the comparability of outcome risk for patients initiating different osteoporosis medications and lipid-lowering therapies, illustrating several ways in which confounding may result. In these studies, NCO methods were implemented in claims-based data sources, with the results used to guide the decision to proceed with comparative effectiveness or safety analyses.Results: Based on this research, we provide recommendations for future NCO studies, including considerations for the identification of confounding mechanisms in the target patient population, the selection of NCOs expected to satisfy required assumptions, the interpretation of NCO effect estimates, and the mitigation of uncontrolled confounding detected in NCO analyses. We propose the use of NCO studies prior to initiating comparative effectiveness or safety research, providing information on the potential presence of uncontrolled confounding in those comparative analyses.Conclusions: Given the increasing use of non-randomized designs for regulatory decision-making, the application of NCO methods will strengthen study design, analysis, and interpretation of real-world data and the credibility of the resulting realworld evidence.

Using Negative Control Populations to Assess Unmeasured Confounding and Direct Effects

“…14,15 One way to mitigate the problem is to prespecify ranges of estimates that are considered to be negligible associations. 40…”

Staging and clean room: Constructs designed to facilitate transparency and reduce bias in comparative analyses of real‐world data

“…30 Negative control outcomes can be chosen based on their hypothesized ability to identify the presence of residual confounding in conceptual domains that are likely to be inadequately controlled. 31 Previous studies have used negative controls in domains that include confounding by indication, functional status, health-seeking behaviors, and access to health care (Table 2). 32,33 During the study development phase, the investigators may convene an external committee comprising researchers and clinicians to finalize the selection of negative control outcomes.…”

“…Negative control outcomes can be chosen based on their hypothesized ability to identify the presence of residual confounding in conceptual domains that are likely to be inadequately controlled 31 . Previous studies have used negative controls in domains that include confounding by indication, functional status, health‐seeking behaviors, and access to health care (Table 2).…”

Confounding and Negative Control Methods in Observational Study of SARS-CoV-2 Vaccine Effectiveness: A Nationwide, Population-Based Danish Health Registry Study

“…Negative control methods have been proposed as an approach to assessing the effects of such biases. 17 A negative control outcome is one that cannot plausibly be caused by the exposure of interest yet shares the same confounding structure as that of the vaccine with respect to association with the primary outcome. 18 , 19 Herein, we used negative control methods to assess the likelihood of confounding in assessment of the effects of SARS-CoV-2 vaccination on infection.…”