Systematic differences in treatment effect estimates between propensity score methods and logistic regression

Martens, Edwin P.; Pestman, Wiebe R.; Boer, Anthonius de; Belitser, Svetlana V.; Klungel, Olaf H.

doi:10.1093/ije/dyn079

Cited by 110 publications

(110 citation statements)

References 28 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…These methods produced similar results and were also approximately equally precise. These findings correspond to previous studies indicating that these methods give approximately the same results [10,[27][28][29]. Propensity score methods can be useful to reduce the number of covariates to be included in a multivariable model in case of limited sample size.…”

Section: Discussionsupporting

confidence: 90%

Impact of influenza vaccination on mortality risk among the elderly

Groenwold¹,

Hoes²,

Hak³

2009

Eur Respir J

View full text Add to dashboard Cite

Estimates of influenza vaccine effectiveness have mostly been derived from nonrandomised studies and therefore are potentially confounded. The aim of the current study was to estimate influenza vaccine effectiveness in preventing mortality among the elderly, taking both measured and unmeasured confounding into account.Information on patients aged o65 yrs from the computerised Utrecht General Practitioner database on eight influenza epidemic periods and summer periods was pooled to estimate influenza vaccine effectiveness in preventing mortality. Summer periods (during which no effect of vaccination was expected) were used as a reference to control for unmeasured confounding in epidemic periods.After adjustment for measured confounders using multivariable regression analysis, propensity score matching and propensity score regression analysis, influenza vaccination reduced mortality risk (odds ratios (ORs) 0.58 (95% confidence interval (CI) 0.46-0.72), 0.56 (95% CI 0.44-0.71) and 0.56 (95% CI 0.45-0.69), respectively). After additional adjustment for unmeasured confounding (as observed during summer periods), the association between influenza vaccination and mortality risk decreased (OR 0.69 (95% CI 0.52-0.92)).We conclude that after state-of-the-art adjustment for typical confounders such as age, sex and comorbidity status, unmeasured confounding still biased estimates of influenza vaccine effectiveness. After taking unmeasured confounding into account, influenza vaccination is still associated with substantial reduction in mortality risk.

show abstract

Section: Discussionsupporting

confidence: 90%

Impact of influenza vaccination on mortality risk among the elderly

Groenwold¹,

Hoes²,

Hak³

2009

Eur Respir J

View full text Add to dashboard Cite

show abstract

“…However, two studies 64,67 also came to the opposite conclusion, that estimates obtained from regression analysis and propensity scoring differ significantly. One simulation study 65 comparing the two methods considered propensity scoring to be the superior method, whereas another study 62 found that propensity scoring is superior when the number of events per confounder is low. The disparate results of these studies means that it is difficult to draw conclusions regarding the relative performance of the different approaches, but Kurth et al 64 make an important observation that potentially explains these different results.…”

Section: Effectiveness Of Adjustment Methodsmentioning

confidence: 99%

“…A summary of the studies that have looked at methods of adjustment for confounding bias in NRSs and how reliable they are [62][63][64][65][66][67][68][69][70] is presented in Appendix 3. Overall, it is unclear which methods are most appropriate in certain circumstances and further research is needed.…”

Section: Adjustment For Bias In Non-randomised Studiesmentioning

confidence: 99%

“…In addition to the seven empirical studies identified, [62][63][64][65][66][67]69 a discussion paper by Biondi-Zoccai et al 70 provides a useful overview of the alternative methods of adjustment. Biondi-Zoccai et al 70 concluded that there is no clearly superior method, noting that 'both standard multivariable methods and propensity scores have key limitations, and none is able to take into account unknown confounders' (p. 736).…”

Section: Propensity Scoring Compared With Instrumental Variables Analmentioning

confidence: 99%

See 1 more Smart Citation

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Hettle

Corbett

Hinde

et al. 2017

Health Technol Assess

106

143

View full text Add to dashboard Cite

BackgroundThe National Institute for Health and Care Excellence (NICE) commissioned a ‘mock technology appraisal’ to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal.MethodsOur research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia.ResultsAn assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the ‘curative’ and ‘bridging to stem-cell transplantation’ treatment approaches separately. Within each TPP, three ‘hypothetical’ evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision.ConclusionsIt is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further.FundingThe National Institute for Health Research Health Technology Assessment programme.

show abstract

“…30,31 Recent research, however, has shown that PSM gives more accurate estimates of marginal treatment effects than traditional methods and that, in certain circumstances, the differences between the two approaches can be substantial. 32 One major advantage of PSM is that, unlike traditional regression methods, cases are only included in the analyses if satisfactory matching can be achieved. All analyses were undertaken using the PSMATCH2 program written for Stata.…”

Section: Approach To Analysismentioning

confidence: 99%

Lower well‐being of young Australian adults with self‐reported disability reflects their poorer living conditions rather than health issues

Emerson

Llewellyn

Honey

et al. 2012

Australian and New Zealand Journal of Public Health

View full text Add to dashboard Cite

Objective: To determine the extent to which the lower well-being of young Australians with disabilities could be accounted for by increased rates of exposure to adversity and reduced access to personal, economic, social and community resources. Results: Self-reported disability was associated with significantly lower scores on all indicators of psychological wellbeing. However, people self-reporting disability were more likely to be exposed to adversity and less likely to have access to a range of personal, economic, material, social and community resources. When these between-group differences in social context were controlled for, the betweengroup differences in psychological wellbeing were largely eliminated. Conclusion:Our results suggest that, among younger adults in Australia, the association between disability and lower psychological well-being largely reflects their increased risk of exposure to adversity and reduced access to resources, rather than the presence of health conditions or impairments per se. Implications: Public health interventionsaimed at improving the well-being of young adults with a disability need to address the predominantly social determinants of wellbeing in this group.

show abstract

Systematic differences in treatment effect estimates between propensity score methods and logistic regression

Abstract: Propensity score methods give in general treatment effect estimates that are closer to the true marginal treatment effect than a logistic regression model in which all confounders are modelled.

Cited by 110 publications

References 28 publications

Impact of influenza vaccination on mortality risk among the elderly

Impact of influenza vaccination on mortality risk among the elderly

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

Lower well‐being of young Australian adults with self‐reported disability reflects their poorer living conditions rather than health issues

Contact Info

Product

Resources

About