CVS Health and the Imaginary Worlds of the Institute for Clinical and Economic Review (ICER)

2019

Self Cite

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is twofold: first, to review the latest report by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) on standards and value claims for modeling imaginary cost-per-quality life year (QALY) worlds in health technology assessments and (ii) given ICER’s commitment to ISPOR standards, to consider the credibility of ICER’s QALYs. The concern is that ICER gives the impression that there is a common and agreed utility metric to support QALY constructs and that there is an agreed standard for creating QALYs within the imaginary reference case meme. The purpose of this commentary is to emphasize that there is no ‘gold standard’ QALY, let alone a ‘gold standard’ utility metric. A QALY is what you assume the QALY should be; it is an artificial construct which allows any number of competing QALYs to be generated for the same target population under different imaginary constructs. Similar objections apply to the application of willingness-to-pay thresholds to support ICER’s business case as the sole arbiter, in pole position, of health technology assessments to support pricing recommendations. In the US. A threshold has to be defined for the QALYs and costs captured, by assumption, within the model. Even for the same target patient population a $50,000 cost per QALY threshold will yield different pricing recommendations depending upon the assumptions driving the QALY estimate, such as choice of utility metric, time spent by stage of disease, frequency and severity of adverse events and costs. While this is an obvious point, it appears not to occur to those who, with a limited technical understanding of how the ICER model is constructed (and ICER is not given to extended technical explanations) take at face value the ICER imaginary construct and the QALY value judgements. Of course, it is also beside the point that the ICER reference case imaginary world was never intended to support empirical assessments of the claims made; we have no idea of whether it is right, if it is wrong and, over the lifetime perspective, we will never know and were never intended to know. Article Type: Commentary

Section: Discussionmentioning

confidence: 99%

ICER, ISPOR AND QALYs: A Tale of Imaginary Worlds

2019

Self Cite

“…This is exemplified in the plethora of physician authored PRO instruments, the overwhelming majority of which fail the standards for of fundamental measurement in assessing response to therapy 28 . But these physicians are not alone, CVS in is adoption of ICER thresholds to determine formulary access also shows a singular lack of awareness of the standards of normal science 29 30 .…”

Section: Beyond Icer Pseudosciencementioning

confidence: 99%

Medicaid Formulary Decisions and the Institute for Clinical and Economic Review: Abandoning Pseudoscience in Imaginary Pharmaceutical Pricing Claims

2021

Self Cite

Medicaid formulary committees and other gatekeepers face a difficult task. On the one hand they can utilize technical expertise in evaluating the real world evidence for clinical, quality of life and resource utilization claims for competing products while on the other hand they may be asked to assess claims built by simulation models for pricing and product access. A common option has been to take modeled claims from third parties such as the Institute for Clinical and Economic Review (ICER) at face value without challenging the model structure, its assumptions and its incremental cost-per-QALY claims set against competing products or the existing standard of care. Unfortunately, from the available evidence, it seems clear that many formulary assessment groups, last but not least those for whom the ICER modeling claims are targeted, have little if any appreciation of the limitations of ICER modeling. There are two substantive issues: (i) a failure to appreciate the limitations imposed by the standards of normal science for credible, empirically evaluable and replicable product claims and (ii) an understanding of limitations imposed by the axioms of fundamental measurement. In the latter case, a failure to recognize that the quality adjusted life year (QALY) is an impossible mathematical construct (hence the I-QALY). To these limitations should be added the potential for constructing competing imaginary claims. Surprisingly, ICER has provided the ideal opportunity to construct competing claims with the launch in late 2020 of the ICER Analytics cloud platform. Formulary committees and other health decision makers should be aware that claims based on the ICER Analytics platform together with competing lifetime modelled claims all fail the standards of normal science. Factoring these into formulary decisions is not only misguided but may have unintended consequences for pricing and access that may disadvantage significantly patients and caregivers. We have spent too much time debating the merits or otherwise of the I-QALY for targeted patient groups with the parties failing to recognize that the focus on simulated cost-per-I-QALY value assessments is a mathematical folly; I-QALY claims are a chimera. The I-QALY, at long last, should be abandoned together with modelled lifetime simulations. Medicaid formulary decision makes should rethink the required evidence base for formulary decisions and negotiations. Care should be taken to revisit previous negotiations where ICER recommendations have been utilized to support pricing and access.

“…Non-physical attributes such as needs-based QoL present a challenge, but one that was met some 60 years ago. This is in obvious contrast to the construction of imaginary or fantasy lifetime reference case worlds which lack any pretense to set the stage for hypothesis testing, relying instead on the weak defense that they provide ‘approximate information’ (or disinformation) which might possibly be of interest to decision makers, or the more credulous, as is the case with CVS and ICER recommendations 51 .…”

Section: Assuming Interval Measurementmentioning

confidence: 99%

Nonsense on Stilts – Part 1: The ICER 2020-2023 Value Assessment Framework for Constructing Imaginary Worlds

2020

Self Cite

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing and access by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to provide a critique of the recently released ICER 2020 Value Assessment Framework (VAF). Although ICER has taken upon itself the pole position in health technology assessments and recommendations for product pricing in the US health care system, the incremental, lifetime cost-per-QALY modeling methodology should not be taken seriously. The creation of imaginary modeled worlds, built entirely from assumption, fails the demarcation test between science and pseudoscience. The ICER evidence reports are best seen as the health technology assessment equivalent of ‘intelligent design’ in counterpoint to ‘natural selection’. It is surprising, therefore, that health care decision makers should take ICER’s recommendations seriously as providing ‘approximate information’ for formulary decision making. What is not appreciated is that the claims made by ICER lack credibility, are impossible to evaluate and lack the ability to be replicated across treatment settings. Indeed, the models presented under the guise of a ‘state of the art’ value assessment were never intended to support evaluable claims. We have no idea and will never know if they are right or if they are wrong. ICER’s position becomes even more untenable once the models presented are assessed in detail. Without in any way supporting the ICER methodology, it is worth noting that all too often ICER’s claims for incremental QALYs in specific models are based upon what appears to be, from the limited evidence presented, a casual and ad hoc assemblage of utility scores from diverse constructs. This is a critical weakness given the role attributed by ICER to the modeled cost-per-QALY claims as central to ICERs imaginary value assessment. ICER also overlooks the fact that the utility scores it captures from the literature to populate its imaginary reference case world lack objectivity. They are ordinal rather than interval measures. To apply these manifest scores to time spent in a disease stage and then aggregate these over different disease stages is nonsensical. The critical issue is one of instrument development. The case made here is for the application of Rasch Measurement Theory (RMT) to construct a unidimensional instrument with interval properties, in this case from the needs fulfillment construct of quality of life (QoL). Unless an instrument meets RMT standards in its development, the logic of Rasch modeling to achieve fundamental measurement standards means that other scales are, by definition, ordinal. It is absurd to ‘assume’ they are interval. RMT is designed to create instruments to evaluate change and test hypotheses. In the absence of instruments that have RMT properties, the cost-per-QALY reference case modelling meme collapses. It is an analytical dead end. If we are to support a meaningful scientific program to discover new facts to support health care delivery and improve the lives of patients, caregivers and their families, then ICER should be put to one side. Article Type: Commentary