Assessing the performance of health technology assessment (HTA) agencies: developing a multi-country, multi-stakeholder, and multi-dimensional framework to explore mechanisms of impact

Abstract

Background Health technology assessment (HTA) agencies have an important role to play in managing the rising demands on health systems. However, creating and running such agencies potentially diverts resources from frontline services. A large number of studies address the question of ‘what is the impact of HTA?’. Several points of heterogeneity in this literature include: purpose of the study, definition of HTA, definition of impact, and scope and rigour of evaluations. Our study seeks to address several limitations in this literature. This study aims to explore the mechanisms of impact of an HTA agency. In doing so, we consider HTA as an institution rather than a knowledge product to build an impact evaluation framework from an international, multi-stakeholder and multi-dimensional perspective. Methods We conducted 9 key informant interviews with experts from the international HTA community. We addressed several questions, informed by existing frameworks of impact within the literature, to understand their perspectives on the mechanisms of impact of an HTA agency. We analyse data using logic modelling and impact mapping, as tools to understand and visualise mechanisms of change. Findings Our impact mapping highlights several distinct, but not necessarily mutually exclusive, mechanisms through which the overall impact of an HTA agency is achieved. These are: the effective conduct of HTA studies; effective use of HTA in agenda-setting and policy formulation processes; effective engagement and external communications; good institutional reputation and fit within the healthcare and policy-making system; effective use of HTA as a tool for the negotiation of health technology prices; and the effective implementation of policy change regarding health technologies. We also identify indicators of these effects. Conclusions Our findings and resulting evaluation framework complement and add to existing literature by offering a new perspective on the mechanisms by which HTA agencies generate impact. This new perspective considers HTA as an institution rather than a knowledge product, is international, multi-dimensional, and includes multi-stakeholder views. We hope the analysis will be useful to countries interested in managing HTA performance.

A situational and stakeholder analysis of health technology assessment in Zimbabwe

“…This reflects an important area of weakness in the current healthcare decision making in Zimbabwe. HTA is characterized by explicit and predetermined frameworks used to determine the value of health services and can be useful for enhancing transparency (40;41). In addition to transparency, allocative efficiency and improving the quality of healthcare were also identified as important attributes of HTA.…”

Economic Rationality in Decision-Making Regarding Newborn Screening: A Case Study in Quebec

“…In high-resource countries, independent health technology assessment (HTA) agencies assess the availability and relevance of evidence to inform decision-makers on which health interventions to offer to the population, how to price them, and how to reimburse them [ 4 ]. HTA agencies aim to ensure that public healthcare systems achieve their fundamental objectives of not only using resources in the most efficient and effective way, i.e., producing as much health as possible with the resources invested in the healthcare system, but of also answering social preoccupations such as equity or social justice [ 5 , 6 ]. The province of Quebec has its own HTA agency, functioning independently from the national HYA agency.…”

Propensity score matching for comparative studies: a tutorial with R and Rex

“…Accurate evaluations of these new treatments are crucial for several reasons: (1) they clarify the benefits of the new technologies and the potential side effects; (2) they update the guidelines to offer better strategies in clinical practice; and (3) they ensure efficient allocation of medical resources [ 5 , 6 ]. Additionally, future medical research can focus on effective technologies and encourage investment in promising ones [ 7 ].…”

“…(2) they update the guidelines to offer better strategies in clinical practice; and (3) they ensure efficient allocation of medical resources [5,6]. Additionally, future medical research can focus on effective technologies and encourage investment in promising ones [7].…”