Accounting for Uncertainty in Health Economic Decision Models by Using Model Averaging

Abstract

Summary. Health economic decision models are subject to considerable uncertainty, much of which arises from choices between several plausible model structures, e.g. choices of covariates in a regression model. Such structural uncertainty is rarely accounted for formally in decision models but can be addressed by model averaging. We discuss the most common methods of averaging models and the principles underlying them. We apply them to a comparison of two surgical techniques for repairing abdominal aortic aneurysms. In model averaging, competing models are usually either weighted by using an asymptotically consistent model assessment criterion, such as the Bayesian information criterion, or a measure of predictive ability, such as Akaike's information criterion. We argue that the predictive approach is more suitable when modelling the complex underlying processes of interest in health economics, such as individual disease progression and response to treatment.Keywords: Akaike's information criterion; Bayesian information criterion; Health economics; Model averaging; Model uncertainty Uncertainty in health economic decision modelsHealth economic decision models are routinely used to guide the choice of the most appropriate treatment for patient groups on the basis of expected benefits and costs, commonly over a lifetime (National Institute for Health and Clinical Excellence, 2008). For chronic and recurring diseases, they are often implemented by using Markov models in which individuals move between clinical states of interest in discrete time periods, and each state is associated with a cost and benefit (Briggs et al., 2006). The parameters of these models include probabilities governing transition between the states, the costs and benefits that are associated with each state and the effects of treatment and other covariates. Ideally, all available relevant evidence is used to inform these parameters, which may include randomized controlled trials and population mortality statistics. However, trials only provide information about relative effectiveness and costs of treatments in the short term, typically 5 years or less. To compare the treatments over patient lifetimes, extrapolations must be made, and the uncertainties that are inherent in the short-term results may be aggravated.The expected costs and benefits for each treatment under the model, which are used to make the decision, are subject to uncertainty (Claxton et al., 2002). In general, decision models are non-linear, so the expected model output does not equal the output evaluated at the expected

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

“…The weights considered in this evaluation were based on AIC scores. As outlined in Jackson et al, 193 the AIC value reported from each survival distribution was converted to a probability weight (w k ) using the following equations:…”

“…To more formally account for the uncertainty surrounding choice of survival distribution, a model-averaging approach was adopted using the methods outlined in Jackson et al 193 This technique involves the parameterisation of uncertainty surrounding the choice of distribution through including all plausible survival functions as part of a weighted distribution and sampling both the parametric uncertainty associated within each distribution and the uncertainty (or weights) surrounding the choice of preferred method. Through the probabilistic analysis, it is therefore possible to estimate the joint distribution of uncertainty around the parameter estimates and the choice of survival function.…”

Use of Implicit and Explicit Bayesian Methods in Health Technology Assessment

“…We envisage that this increase in the uptake will be sustained into the future because, as HTA questions become more complex and demanding, and methodology evolves in response to this, the flexibility of Bayesian methods seem best suited to implement and address non-standard, often complex, approaches. For example, recent methodological developments where there is potential for Bayesian methods to make an even greater impact on healthcare evaluations in the future include i) assessing and adjusting for the relevance and rigour of evidence used in both the evidence syntheses (19); ii) addressing structural uncertainty in the economic decision model (42); iii) assessing model fit in both the evidence syntheses and economic decision model (43); and iv) incorporating beliefs of decision makers. …”

Survival Models in Health Economic Evaluations: Balancing Fit and Parsimony to Improve Prediction

“…This is usually performed by presenting cost-effectiveness under alternative models, but with no formal measures of the plausibility of different assumptions. Jackson et al (2009Jackson et al ( , 2010 described how statistical models used in economic evaluations can be formally compared based on pre-dictive criteria, and used model averaging to account for uncertainty about model selection, although they did not explicitly discuss survival modelling.…”

“…Improvements to the approximation used by DIC have been suggested (Plummer, 2008) for situations when p D is large compared to the sample size, as in the semiparametric models we used. Jackson et al (2009Jackson et al ( , 2010 reviewed methods for accounting for model uncertainty among frequentist and Bayesian models in a health economic setting. One particular principle identified, model averaging based on focused criteria , might be of benefit in the oral cancer application.…”