Confounding can cause substantial bias in nonexperimental studies that aim to estimate causal effects. Propensity score methods allow researchers to reduce bias from measured confounding by summarizing the distributions of many measured confounders in a single score based on the probability of receiving treatment. This score can then be used to mitigate imbalances in the distributions of these measured confounders between those who received the treatment of interest and those in the comparator population, resulting in less biased treatment effect estimates. This methodology was formalized by Rosenbaum and Rubin in 1983 and, since then, has been used increasingly often across a wide variety of scientific disciplines. In this review article, we provide an overview of propensity scores in the context of real-world evidence generation with a focus on their use in the setting of single treatment decisions, that is, choosing between two therapeutic options. We describe five aspects of propensity score analysis: alignment with the potential outcomes framework, implications for study design, estimation procedures, implementation options, and reporting. We add context to these concepts by highlighting how the types of comparator used, the implementation method, and balance assessment techniques have changed over time. Finally, we discuss evolving applications of propensity scores.

New user designs restricting to treatment initiators have become the preferred design for studying drug comparative safety and effectiveness using non-experimental data. This design reduces confounding by indication and healthy adherer bias at the cost of smaller study sizes and reduced external validity, particularly when assessing a newly approved treatment compared to standard treatment. The prevalent new user design includes adopters of a new treatment who switched from or previously used standard treatment (i.e. the comparator), expanding study sample size and potentially broadening the study population for inference. Previous work has suggested the use of time conditional propensity score matching to mitigate prevalent user bias. In this study, we describe three “types” of initiators of a treatment: new users, direct switchers, and delayed switchers. Using these initiator types, we articulate the causal questions answered by the prevalent new user design and compare them to those answered by the new user design. We then show, using simulation, how conditioning on time since initiating the comparator (rather than full treatment history) can still result in a biased estimate of the treatment effect. When implemented properly, the prevalent new user design estimates new and important causal effects distinct from the new user design.

To extend previous simulations on the performance of propensity score (PS) weighting and trimming methods to settings without and with unmeasured confounding, Poisson outcomes, and various strengths of treatment prediction (PS c-statistic), we simulated studies with a binary intended treatment T as a function of 4 measured covariates. We mimicked treatment withheld and last-resort treatment by adding two “unmeasured” dichotomous factors that directed treatment to change for some patients in both tails of the PS distribution. The number of outcomes Y was simulated as a Poisson function of T and confounders. We estimated the PS based on measured covariates and trimmed the tails of the PS distribution using three strategies (“Crump”, “Stürmer”, and “Walker”). After trimming and re-estimation, we used alternative PS weights to estimate the treatment effect (rate ratio): IPTW, SMR-treated, SMR-untreated, overlap (ATO), matching, and entropy. With no unmeasured confounding, ATO (123%) and “Crump” trimming (112%) improved relative efficiency compared with untrimmed IPTW. With unmeasured confounding, untrimmed estimates were biased irrespective of weighting method and only Stürmer and Walker trimming consistently reduced bias. In settings where unmeasured confounding (e.g., frailty) may lead physicians to withhold treatment, Stürmer and Walker trimming should be considered before primary analysis.

Statins are indicated in patients with elevated levels of high-sensitivity C-reactive protein and normal low-density lipoprotein cholesterol based on results of the multicountry trial, Justification for the Use of Statins in Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER) (2003-2008), but the benefit in real-world populations remains unknown. We sought to generalize JUPITER results to trial-eligible population using data from the UK Clinical Practice Research Datalink (CPRD), 2001-2014. We multiply imputed missing baseline characteristics for the CPRD population and selected the trial-eligible participants as the target population based on observed and imputed values. Trial participants were weighted to be representative of the CPRD population (n = 383,418) based on individual predicted probability of selection into the trial. Trial participants were also standardized to the CPRD population without missing values (n = 2,677). In JUPITER, rosuvastatin reduced cardiovascular risk with a 3-year risk difference of -2.0% (95% confidence interval (CI): -2.9, -1.1). The rosuvastatin effect was muted in the first 2 years but remained strong at 3 years after standardizing to the imputed CPRD population (3-year risk difference = -2.7%; 95% CI: -5.8, 0.4) and the CPRD population without missing data (3-year risk difference = -1.7%; 95% CI: -3.5, 0.1). The study serves as an illustration of possible approaches to understanding generalizability of trials using real-world databases given limitations due to missing data on inclusion/exclusion criteria.