Variable selection for propensity score models when estimating treatment effects on multiple outcomes: a simulation study

Abstract

Purpose It is often preferable to simplify the estimation of treatment effects on multiple outcomes by using a single propensity score (PS) model. Variable selection in PS models impacts the efficiency and validity of treatment effects. However, the impact of different variable selection strategies on the estimated treatment effects in settings involving multiple outcomes is not well understood. The authors use simulations to evaluate the impact of different variable selection strategies on the bias and precision of effect estimates to provide insight into the performance of various PS models in settings with multiple outcomes. Methods Simulated studies consisted of dichotomous treatment, two Poisson outcomes, and eight standard-normal covariates. Covariates were selected for the PS models based on their effects on treatment, a specific outcome, or both outcomes. The PSs were implemented using stratification, matching, and weighting (IPTW). Results PS models including only covariates affecting a specific outcome (outcome-specific models) resulted in the most efficient effect estimates. The PS model that only included covariates affecting either outcome (generic-outcome model) performed best among the models that simultaneously controlled measured confounding for both outcomes. Similar patterns were observed over the range of parameter values assessed and all PS implementation methods. Conclusions A single, generic-outcome model performed well compared with separate outcome-specific models in most scenarios considered. The results emphasize the benefit of using prior knowledge to identify covariates that affect the outcome when constructing PS models and support the potential to use a single, generic-outcome PS model when multiple outcomes are being examined.

Chasing Balance and Other Recommendations for Improving Nonparametric Propensity Score Models

“…This finding is consistent with what others in the field have found (Brookhart et al 2006, Wyss et al 2013, Setodji et al Submitted , Setodji et al conditional acceptance ). Given that we include such a large number of variables into the model, it is notable that all of the methods still perform reasonably well in all cases where distractor variables are added into the model (maximum mean ASMDs and standardized bias in the treatment effect estimate are all below 0.11), except the ATE case in the abstainers data generation case study.…”

“…However, studies on parametric methods have shown that including covariates that are unrelated to the treatment variable when estimating propensity scores can lead to weights with greater variability and poorer balance for covariates correlated with the treatment indicator (Brookhart et al 2006, Wyss et al 2013). This, in turn, increases the bias and decreases the efficiency of the treatment effect estimate.…”

Preoperative Statin Administration Does Not Protect Against Early Postoperative Acute Respiratory Distress Syndrome

“…statins), but also those relating to the outcome of interest (i.e. ARDS) (32,33). Including this more comprehensive list of variables increases the precision of the estimated exposure effect without increasing bias (34).…”

Propensity Scores in Pharmacoepidemiology: Beyond the Horizon

“…When several are of interest, simulation results suggest that a generic propensity score model based on their shared confounders performs nearly as well as separate models built for each outcome. [42] These algorithms appear promising for variable selection, but have not been studied in depth.…”