A comparison of two methods of estimating propensity scores after multiple imputation

Abstract

In many observational studies, analysts estimate treatment effects using propensity scores, e.g. by matching or sub-classifying on the scores. When some values of the covariates are missing, analysts can use multiple imputation to fill in the missing data, estimate propensity scores based on the m completed datasets, and use the propensity scores to estimate treatment effects. We compare two approaches to implement this process. In the first, the analyst estimates the treatment effect using propensity score matching within each completed data set, and averages the m treatment effect estimates. In the second approach, the analyst averages the m propensity scores for each record across the completed datasets, and performs propensity score matching with these averaged scores to estimate the treatment effect. We compare properties of both methods via simulation studies using artificial and real data. The simulations suggest that the second method has greater potential to produce substantial bias reductions than the first, particularly when the missing values are predictive of treatment assignment.

Carvedilol Compared With Metoprolol Succinate in the Treatment and Prognosis of Patients With Stable Chronic Heart Failure

“…We computed the C-statistic as 0.71 from the logistic regression model with the averaged linear predictor as predictor of caseness and obtained the identical result after averaging over the n=100 C-statistics, computed from each of the imputed samples. This result corroborates the suitability of using the averaged linear predictor in the full original sample for the matching procedure, as proposed by Mitra et al 28 On the basis of the averaged propensity score of the multiple imputed data sets, the matching procedure identified 939 pairs of patients with equal probability of either carvedilol or metoprolol succinate therapy while receiving it at equivalent doses. Of these, 530 patients died during follow-up.…”

“…We identified N-terminal pro-brain natriuretic peptide, hemoglobin, and loop diuretic dose as variables with a significant amount of missing values ( Table 1 Following the study by Mitra et al, 28 the calculation of the propensity score was repeated in each of the multiple imputed data sets (n=100), and the propensity score was averaged for each record across the completed data sets. We computed the C-statistic as 0.71 from the logistic regression model with the averaged linear predictor as predictor of caseness and obtained the identical result after averaging over the n=100 C-statistics, computed from each of the imputed samples.…”

“…Then the multivariable calculations were repeated in the multiple imputed data set. Moreover, following the study by Mitras et al, 28 the calculation of the propensity score was repeated in each of the multiple imputed data sets. The propensity score was averaged for each record across the completed data sets.…”

Transplantation for Idiopathic Pulmonary Arterial Hypertension

“…25 Multivariable logistic regression models that included all available prelisting or pretransplantation variables were used to develop a propensity score for all patients in each of the 4 previously described analyses; to handle missing data, propensity scores were calculated across all imputed data sets using the across approach described by Mitra. 20,26 We next performed a 1:1 nearest-neighbor matching algorithm without replacement (using a caliper of 0.25 of the standard deviation of the linear propensity score); balance was achieved in our model by the standardized differences approach. Values are n (%) or mean±SD.…”

Family formation patterns and physical health in Australia: a life course approach

“…As the data are multiply imputed, the propensity score is estimated separately within each imputed dataset, and the resulting m estimates are averaged across estimations for each individual to give a single estimate for use in subsequent analysis. This sequence was chosen on the basis of a simulation study by Mitra and Reiter (2016), who found that it produced marginally greater bias reduction than alternative strategies (such as estimating the final models using different estimates of the propensity score within each imputed dataset, and averaging the resulting parameter estimates) when combining MICE with propensity score methods.…”