Trusting the Results of Model-Based Economic Analyses: Is there a Pragmatic Validation Solution?

Abstract

Models have become a nearly essential component of health technology assessment. This is because the efficacy and safety data available from clinical trials are insufficient to provide the required estimates of impact of new interventions over long periods of time and for other populations and subgroups. Despite more than five decades of use of these decision-analytic models, decision makers are still often presented with poorly validated models and thus trust in their results is impaired. Among the reasons for this vexing situation are the artificial nature of the models, impairing their validation against observable data, the complexity in their formulation and implementation, the lack of data against which to validate the model results, and the challenges of short timelines and insufficient resources. This article addresses this crucial problem of achieving models that produce results that can be trusted and the resulting requirements for validation and transparency, areas where our field is currently deficient. Based on their differing perspectives and experiences, the authors characterize the situation and outline the requirements for improvement and pragmatic solutions to the problem of inadequate validation.

R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study

“…When an analysis can be programmed in Excel and remain 'fit for purpose' in terms of computational efficiency and analytic transparency, there seems little benefit in moving to an alternative approach. Moving forward, increasing acceptance by external reviewers will be driven by increased training as part of health economics courses and using accepted and validated code for as many functionalities as feasible, with new analyses being well highlighted and documented [58].…”

An Educational Review About Using Cost Data for the Purpose of Cost-Effectiveness Analysis

“…the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement [12], which provides guidance on how to report economic evaluations more consistently and with transparency, and the Philips et al [13] checklist based on a review of guidelines for good practice in decision-analytic modelling for health technology assessment (HTA). Other authors have also suggested pragmatic steps to improve the validation of models [14].…”

Quantitative Evidence Synthesis Methods for the Assessment of the Effectiveness of Treatment Sequences for Clinical and Economic Decision Making: A Review and Taxonomy of Simplifying Assumptions

“…Our review identified some good examples where a model was further developed over time to address multiple reimbursement decisions (e.g. Birmingham Rheumatoid Arthritis Model [BRAM] [75,158], Tran-Duy model [68,122], Sheffield rheumatoid arthritis models [159], and the Advanced Simulation Model [78]). However, each was developed by the same research group.…”

“…In some models, when patients move quickly through the sequence of treatments (for example, early discontinuation due to adverse effects), simulated patients can actually benefit from having multiple 'short-term' benefits from different treatments, thus gaining an additive effect. Some included models of inflammatory arthritis attempted to overcome this problem by introducing a 'rebound' effect, which automatically returns the patient to their starting severity (used in, for example, the Diamantopoulos model [89]), or following some natural, background increase (as used in the BRAM [158]). Although the evidence to support this type of assumption is weak, it is arguably better than the false benefits generated by models otherwise.…”