The Comparison of Matching Methods Using Different Measures of Balance: Benefits and Risks Exemplified within a Study to Evaluate the Effects of German Disease Management Programs on Long‐Term Outcomes of Patients with Type 2 Diabetes

Abstract

Objective. To present a case study on how to compare various matching methods applying different measures of balance and to point out some pitfalls involved in relying on such measures. Data Sources. Administrative claims data from a German statutory health insurance fund covering the years 2004-2008. Study Design. We applied three different covariance balance diagnostics to a choice of 12 different matching methods used to evaluate the effectiveness of the German disease management program for type 2 diabetes (DMPDM2). We further compared the effect estimates resulting from applying these different matching techniques in the evaluation of the DMPDM2. Principal Findings. The choice of balance measure leads to different results on the performance of the applied matching methods. Exact matching methods performed well across all measures of balance, but resulted in the exclusion of many observations, leading to a change of the baseline characteristics of the study sample and also the effect estimate of the DMPDM2. All PS-based methods showed similar effect estimates. Applying a higher matching ratio and using a larger variable set generally resulted in better balance. Using a generalized boosted instead of a logistic regression model showed slightly better performance for balance diagnostics taking into account imbalances at higher moments. Conclusion. Best practice should include the application of several matching methods and thorough balance diagnostics. Applying matching techniques can provide a useful preprocessing step to reveal areas of the data that lack common support. The use

An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma

“…He does not include exact matching, however, and does not investigate the impact of the method choice on the primary endpoint of interest. Studies by Wells et al12 and Fullerton et al13 compare matching methods in real data sets, including exact matching, and discuss the strengths and weaknesses of each. Despite these useful studies, it is important to gain more evidence in this area, to strengthen the conclusions that are drawn from the analyses of one data set and to allow investigators to make more informed decisions on the design of observational studies.…”

Logistic Regression and Generalized Boosted Modelling in Inverse Probability of Treatment Weighting: A Simulation and Case Study of Outpatients with Coronary Heart Disease

“…Therefore, the main purpose of this study is to compare the ability of logistic regression and GBM to estimate the treatment effect and covariate balance in propensity score weighting under linear or nonlinear conditioning models. Although Jacqueline M Burgette, et al, Xin M, Fullerton B and other researchers [39][40][41] compared logistic regression with GBM, they did not consider the interaction effect and quadratic relationships among the treatment variable and covariates. Moreover, this study assessed the covariate balance and distribution of weights to make the results more reliable.…”

Exact matching of trabectome-mediated ab interno trabeculectomy to conventional trabeculectomy with mitomycin C followed for 2 years

“…Limitations of our study include its retrospective nature. This shortcoming is countered by exact matching, a robust automated statistical method that creates a highly balanced comparison and reduces confounding [ 59 , 60 ]. Although the algorithm discarded AIT data, we were fortunate to be able to afford this and create identical IOP and glaucoma eye drop matches with the much more limited number of TRAB so that only 31 were lost.…”

“…Although the algorithm discarded AIT data, we were fortunate to be able to afford this and create identical IOP and glaucoma eye drop matches with the much more limited number of TRAB so that only 31 were lost. Had we increased the number of matching variables to improve the precision of matching, many more patients would have been excluded, reducing the study sample size and variability of the patient population [ 60 , 61 ]. Consequently, we applied nearest neighbor matching to age, and AIT patients ended up with a younger average age of 50 ± 15 years compared to TRAB with 67 ± 11 years.…”