State of the ART? Two New Tools for Risk Communication in Health Technology Assessments

Abstract

Purpose Outcomes of health technology assessments (HTA) are uncertain, and decision-making is associated with a risk. This risk, consisting of the probability of making a wrong decision and its impact, is rarely considered in HTA. This hampers transparent and consistent risk assessment and management. The aim of this study was to develop risk communication tools in the context of health technology decision-making under uncertainty. Methods We performed a scoping review of tools for uncertainty and risk communication within HTA using citation pearlgrowing. We developed two tools, drawing on existing publications on risk and uncertainty communication for inspiration. Individual semi-structured interviews with HTA stakeholders were performed to identify potential improvements in usefulness, user-friendliness, and information adequacy. Tools were amended and further evaluated in a real-world HTA and workshop with HTA stakeholders. Results The identified risk communication tools did not include non-quantified uncertainties, and did not link to risk management strategies. We developed two tools: the Assessment of Risk Table (ART), for a summary of quantified and nonquantified uncertainties and the resulting risk assessment, and the Appraisal of Risk Chart (ARCH), for linking net benefit and risk outcomes to appropriate risk management strategies. Stakeholders appreciated the usefulness of the tools. They also highlighted that more information on local policy options was required for optimal risk management use, and HTA processes may need adapting. ConclusionThe risk communication tools presented here can help assess risk, facilitate communication between analysts and decision-makers, and guide the appropriate use of available risk management strategies.

Comprehensive Review of Methods to Assess Uncertainty in Health Economic Evaluations

“…Assessment of Risk Table (ART) [67] Table to summarise cost effectiveness, uncertainty and opportunity loss information Assessment of Risk Chart (ARCH) [67] Chart to link opportunity loss to appropriate risk management strategies and decision frameworks has been widely endorsed as a reporting tool for HEEs. The detail of the items of CHEERS makes it easy to use for less experienced analysts and improves reproducibility.…”

“…Additionally, further research is necessary to analyse and communicate uncertainty that is not quantified. We found only two methods to analyse uncertainty that is not quantified (expert elicitation, threshold analyses [ 1 ]) and three methods to communicate uncertainty that is not quantified (the threshold analysis figure [ 1 ], the ART [ 67 ] and the ARCH [ 67 ]). Threshold analyses and the relating figure result in an estimate of how sensitive model outcomes are, when uncertainty is not quantified.…”

Uncertainty management in regulatory and health technology assessment decision-making on drugs: guidance of the HTAi-DIA Working Group

“…Uncertainty management refers to carefully balancing the benefits of reducing uncertainty with the efforts of doing so and the opportunity costs of the alternatives (2). The added value of reducing uncertainty has many dimensions, for example, improved knowledge on treatment efficacy or safety, a reduced possibility for decision errors, a better understanding of the relative effectiveness or associated costs reducing the risk of healthcare displacement or efficiency in the design and implementation of solutions (2;6;10–13). The costs associated with uncertainty reduction in terms of additional evidence generation can be tackled to some extent through value of information (VOI) analyses (14).…”

Response to “UNCERTAINTY MANAGEMENT IN REGULATORY AND HEALTH TECHNOLOGY ASSESSMENT DECISION-MAKING ON DRUGS: GUIDANCE OF THE HTAi-DIA WORKING GROUP”

“…There is an opportunity to build upon the challenges other researchers in the area of uncertainty assessment in and outside of HTA have identified and the methods proposed to address these. The progress made on the following topics has not been sufficiently covered in the guidance, including but not limited to: uncertainty identification, for example, using the TRUST tool (11) and other methods (6;18); uncertainty analysis methods including Bayesian methods (23), value of information (24–26), structured expert elicitation (27;28), and incorporation of difficult to quantify uncertainty (29–33); uncertainty communication (17;34–36); link between uncertainty and evaluation of managed entry agreement (MEA) schemes (36–39); uncertainty (in)tolerance in regulatory and HTA decision-making (3;40–42). …”

“…As a next step, the Working Group refers to the link of their proposed framework with mitigation strategies. Importantly, there are existing frameworks and tools covering this topic including frameworks for classifications of different MEA schemes (43;44), and approaches for assessing MEAs (36;39;45). We urge the Working Group to consider and transparently build upon these, where relevant.…”