Identification of treatment effects in Health Economics

Many cost-effectiveness analyses (CEAs) use data from observational studies. Statistical methods can only address selection bias if they make plausible assumptions. No quality assessment tool is available for appraising CEAs that use observational studies. We developed a new checklist to assess statistical methods for addressing selection bias in CEAs that use observational data. The checklist criteria were informed by a conceptual review and applied in a systematic review of economic evaluations. Criteria included whether the study assessed the 'no unobserved confounding' assumption, overlap of baseline covariates between the treatment groups and the specification of the regression models. The checklist also considered structural uncertainty from the choice of statistical approach. We found 81 studies that met the inclusion criteria: studies tended to use regression (51%), matching on individual covariates (25%) or matching on the propensity score (22%). Most studies (77%) did not assess the 'no observed confounding' assumption, and few studies (16%) fully considered structural uncertainty from the choice of statistical approach. We conclude that published CEAs do not assess the main assumptions behind statistical methods for addressing selection bias. This checklist can raise awareness about the assumptions behind statistical methods for addressing selection bias and can complement existing method guidelines for CEAs.

show abstract

“…These considerations might be complemented with placebo tests (Jones, 2007), which can detect violations of the assumption.…”

Section: Step 2: Applying the Checklistmentioning

confidence: 99%

Statistical Methods for Cost‐effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool and Review of Current Practice

2012

View full text Add to dashboard Cite

show abstract

“…Studies using observational data are prone to selection bias, with the treatment effect potentially being confounded with individual, provider or other characteristics [3]. Selection bias constitutes a major threat to the internal validity of a study and unless its presence can be minimised the estimated treatment effects do not necessarily imply a cause and effect relationship [4]. …”

Section: Introductionmentioning

confidence: 99%

Do health economic evaluations using observational data provide reliable assessment of treatment effects?

Rovithis

2013

Health Econ Rev

View full text Add to dashboard Cite

Economic evaluation in modern health care systems is seen as a transparent scientific framework that can be used to advance progress towards improvements in population health at the best possible value. Despite the perceived superiority that trial-based studies have in terms of internal validity, economic evaluations often employ observational data. In this review, the interface between econometrics and economic evaluation is explored, with emphasis placed on highlighting methodological issues relating to the evaluation of cost-effectiveness within a bivariate framework. Studies that satisfied the eligibility criteria exemplified the use of matching, regression analysis, propensity scores, instrumental variables, as well as difference-in-differences approaches. All studies were reviewed and critically appraised using a structured template. The findings suggest that although state-of-the-art econometric methods have the potential to provide evidence on the causal effects of clinical and policy interventions, their application in economic evaluation is subject to a number of limitations. These range from no credible assessment of key assumptions and scarce evidence regarding the relative performance of different methods, to lack of reporting of important study elements, such as a summary outcome measure and its associated sampling uncertainty. Further research is required to better understand the ways in which observational data should be analysed in the context of the economic evaluation framework.

show abstract

“…For many CEA, RCTs are insufficient, and for some decision problems, the only evidence for comparing treatment alternatives is from non‐randomized studies (NRS), for example, natural experiments (Deeks et al , ). When treatment assignment is non‐random, the treatment groups are drawn from different populations, and failure to correct for the resulting baseline differences can lead to biased estimates (Basu et al , , ; Jones, ). CEA require methods that fully adjust for imbalances between the groups.…”

Section: Introductionmentioning

confidence: 99%

A matching method for improving covariate balance in cost‐effectiveness analyses

Sekhon

Grieve

2011

Health Economics

View full text Add to dashboard Cite

In cost-effectiveness analyses (CEA) that use randomized controlled trials (RCTs), covariates of prognostic importance may be imbalanced and warrant adjustment. In CEA that use non-randomized studies (NRS), the selection on observables assumption must hold for regression and matching methods to be unbiased. Even in restricted circumstances when this assumption is plausible, a key concern is how to adjust for imbalances in observed confounders. If the propensity score is misspecified, the covariates in the matched sample will be imbalanced, which can lead to conditional bias. To address covariate imbalance in CEA based on RCTs and NRS, this paper considers Genetic Matching. This matching method uses a search algorithm to directly maximize covariate balance. We compare Genetic and propensity score matching in Monte Carlo simulations and two case studies, CEA of pulmonary artery catheterization, based on an RCT and an NRS. The simulations show that Genetic Matching reduces the conditional bias and root mean squared error compared with propensity score matching. Genetic Matching achieves better covariate balance than the unadjusted analyses of the RCT data. In the NRS, Genetic Matching improves on the balance obtained from propensity score matching and gives substantively different estimates of incremental cost-effectiveness. We conclude that Genetic Matching can improve balance on measured covariates in CEA that use RCTs and NRS, but with NRS, this will be insufficient to reduce bias; the selection on observables assumption must also hold.

show abstract

Identification of treatment effects in Health Economics

Abstract: No Abstract

Cited by 26 publications

References 43 publications

Statistical Methods for Cost‐effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool and Review of Current Practice

Statistical Methods for Cost‐effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool and Review of Current Practice

Do health economic evaluations using observational data provide reliable assessment of treatment effects?

A matching method for improving covariate balance in cost‐effectiveness analyses

Contact Info

Product

Resources

About