Learning and practicing more value-reflective, problem-setting health technology assessment: experiences and lessons from the VALIDATE project

Abstract

Objectives: To conduct a formative evaluation of applying the VALIDATE approach in practice by (i) assessing how students appreciated the e-learning course, (ii) exploring how, for what purposes and with what outcomes the acquired VALIDATE competences subsequently were used in internships in different institutional contexts, and how this was shaped by these contexts, and (iii) what this shows on real-world use of VALIDATE. Methods: Comparative discussion of experiences of applying the VALIDATE approach via a semistructured survey and oral feedback from e-course students; final reports on internships in health technology assessment (HTA) practice, followed by semistructured interviews with interns and supervisors to complement and interpret results. Results: All students considered the VALIDATE approach an enlightening and important addition to current HTA knowledge, especially regarding understanding the relation between empirical analysis and normative inquiry, identifying policy relevant questions and using the method of reconstructing interpretive frames for scoping. The latter appeared intellectually challenging and requiring some prior HTA knowledge. The use the VALIDATE approach in practice shows that interns productively redefined the HTA problem, based on appreciation of different stakeholders' definition of the issue; they experienced constraints from retrieving all relevant perspectives from existing literature as well as from institutional rules and routines. Conclusions: Some challenges in applying the VALIDATE approach deserve attention for its future use: currently used research approaches often assume a problem as "given"; and the data needed on different perspectives is often not reported in scientific literature. Finally, data gathering on and evaluation of value dimensions was experienced as challenging.Health technology assessment (HTA) practices that separate facts and values by first collecting all facts on a health technology, as a basis for subsequent assessment in terms of values, obscure the value-ladenness of the particular features built into the technology in the first place. Also, the professional logics informing what facts to collect, and which outcome measures are relevant, are "self-evidently" adopted. Recognizing this, the VALIDATE (VALues In Doing Assessments of healthcare Technologies) approach holds that empirical analysis in HTA should always be viewed in conjunction with the interpretive frames (IFs) from which such analysis makes sense (1). An IF is defined as a perspective on a particular issue in a particular context comprising the problem definition and solution assessment of that issue; and generic beliefs that form the lens through which the issue-in-context is perceived: background theories and normative preferences (2).By including stakeholders and taking into account their IFs in HTAs, when considering safety, clinical, economic, and wider social and ethical implications of a health technology, VALIDATE seeks to complement current approaches to HTA in order to reali...

Ethical, Legal, Organisational and Social Issues of Teleneurology: A Scoping Review

“…To incorporate research questions into this study that were relevant to all stakeholders, we introduced the VALIDATE [ 20 , 21 , 22 ] framework early in the review process. In the health research field, the criteria of patients or stakeholders, in general, might differ from those of researchers and healthcare professionals and can add value.…”

“…Special attention has been given to aspects related to the protection of the therapeutic relationship, both for the professional and for the patient, so that it can be developed respecting the ethical principles that govern its treatment [ 20 , 21 , 22 , 30 , 34 , 46 , 58 , 72 ]. The questions proposed by the working group also pointed out the concern for the legal aspects related to the implementation of TN in Spain.…”

“…The main strength of this review has been the contribution made by stakeholders in the initial phase of defining the research question in a virtual consensus meeting, developed as part of the process proposed by the VALIDATE project analysis framework [ 20 , 21 , 22 ]. This has allowed our results to gain relevance, as they reflect the different perspectives and values of the group of stakeholders consulted.…”

“…A detailed protocol for the review was developed by the research team and further approved by the Spanish Ministry of Health (available from the corresponding author on request). The assessment of ethical, legal, organisational and social aspects was carried out by adapting the framework of the Core Model 3.0 of EUnetHTA, the criteria established by the Spanish Network of Agencies for Health Technology Assessment for the National Health Service, and the analysis criteria of the European VALIDATE Project (VALues In Doing Assessments of healthcare TEchnologies) [ 20 , 21 , 22 ]. None of these frameworks addresses patient safety specifically in teleneurology, but rather in general health technology terms.…”

Health technology assessment: A matter of facts and values

“…In addition, it argues that incorporating specific insights and methods from policy sciences offers a concrete way for achieving two important objectives: (i) conducting a scoping exercise as an integral part of HTA, and (ii) involving stakeholders in a more structural and constructive way than the currently is usually the case. The second paper (5) reports the experiences and lessons that were obtained with the VALIDATE teaching program and training materials: how did students experience the E-learning course, and how did it help them in conducting an internship that lived up to the aspirations, as expressed in the course and associated handbook? The paper shows that, overall, ratings of the teaching materials were high, but that students found it challenging to maintain and employ the critical perspective learned in VALIDATE and to justify their choices to the staff of host organizations.…”