Background Showing how engagement adds value for all stakeholders can be an effective motivator for broader implementation of patient engagement. However, it is unclear what methods can best be used to evaluate patient engagement. This paper is focused on ways to evaluate patient engagement at three decision‐making points in the medicines research and development process: research priority setting, clinical trial design and early dialogues with regulators and health technology assessment bodies. Objective Our aim was to review the literature on monitoring and evaluation of patient engagement, with a focus on indicators and methods. Search strategy and inclusion criteria We undertook a scoping literature review using a systematic search, including academic and grey literature with a focus on evaluation approaches or outcomes associated with patient engagement. No date limits were applied other than a cut‐off of publications after July 2018. Data extraction and synthesis Data were extracted from 91 publications, coded and thematically analysed. Main results A total of 18 benefits and 5 costs of patient engagement were identified, mapped with 28 possible indicators for their evaluation. Several quantitative and qualitative methods were found for the evaluation of benefits and costs of patient engagement. Discussion and conclusions Currently available indicators and methods are of some use in measuring impact but are not sufficient to understand the pathway to impact, nor whether interaction between researchers and patients leads to change. We suggest that the impacts of patient engagement can best be determined not by applying single indicators, but a coherent set of measures.

Background Patient engagement is becoming more customary in medicine development. However, embedding it in organizational decision‐making remains challenging, partly due to lack of agreement on its value and the means to evaluate it. The objective of this project was to develop a monitoring and evaluation framework, with metrics, to demonstrate impact and enhance learning. Methods A consortium of five patient groups, 15 biopharmaceutical companies and two academic groups iteratively created a framework in a multi‐phase participatory process, including analysis of its application in 24 cases. Results The framework includes six components, with 87 metrics and 15 context factors distributed among (sub)components: (a) Input: expectations, preparations, resources, representativeness of stakeholders; (b) Activities/process: structure, management, interactions, satisfaction; (c) Learnings and changes; (d) Impacts: research relevance, study ethics and inclusiveness, study quality and efficiency, quality of evidence and uptake of products, empowerment, reputation and trust, embedding of patient engagement; (e) Context: policy, institutional, community, decision‐making contextual factors. Case study findings show a wide variation in use of metrics. There is no ‘one size fits all’ set of metrics appropriate for every initiative or organization. Presented sample sets of metrics can be tailored to individual situations. Conclusion Introducing change into any process is best done when the value of that change is clear. This framework allows participants to select what metrics they value and assess to what extent patient engagement has contributed. Patient contribution Five patient groups were involved in all phases of the study (design, conduct, interpretation of data) and in writing the manuscript.

BackgroundMeaningful patient engagement (PE) in medicines development and during the life cycle of a product requires all stakeholders have a clear understanding of respective expectations.ObjectiveA qualitative survey was undertaken to understand stakeholder expectations.DesignThe survey explored 4 themes from the perspective of each stakeholder group: meaning, views, expectations and priorities for PE. Participants were grouped into 7 categories: policymakers/regulators; health‐care professionals (HCPs); research funders; payers/purchasers/HTA; patients/patient representatives; pharmaceutical/life sciences industry; and academic researchers.ResultsFifty‐nine interviews were conducted across a range of geographies, PE experience and job seniority/role. There was consensus across stakeholders on meaning of PE; importance of promoting PE to a higher level than currently; need for a more structured process and guidance. There was little consensus on stakeholder expectations and roles. Policymakers/regulators were expected by others to drive PE, create a framework and facilitate PE, provide guidelines of good practice and connect stakeholders, but this expectation was not shared by the policymakers/regulators group. HCPs were seen as the link between patients and other stakeholders, but HCPs did not necessarily share this view.Discussion and conclusionsDespite broad stakeholder categories, clear themes emerged: there is no “leader”; no stakeholder has a clear view on how to meaningfully engage with patients; there are educational gaps; and a structure and guidance for PE is urgently required. Given the diversity of stakeholders, there needs to be multistakeholder collaborative leadership. Effective collaboration requires consensus on roles, responsibilities and expectations to synergize efforts to deliver meaningful PE in medicines life cycle.

Background There is increased recognition that incorporating patients’ perspectives and insights into the medicines development process results in better health outcomes and benefits for all involved stakeholders. Despite the increased interest and the existence of frameworks and practical recommendations, patient engagement (PE) is not yet considered standard practice. The objective of this work was to provide a roadmap to support systematic change in all stakeholder organisations involved in medicines development across Europe, patients and patient organisations, medicines developers, academia, regulatory authorities, Health Technology Assessment bodies, payers, policy-makers and public research funders, to sustain PE practices. Methods A mixed-methods approach was used by the EU-funded Innovative Medicines Initiative PARADIGM Consortium to co-develop the sustainability roadmap including background work to identify success factors and scenarios for sustainable PE. The roadmap development was based on the Theory of Change concept and populated with findings from (1) interviews with national/ and international institutions with the potential to increase PE uptake by other stakeholders; (2) multi-stakeholder workshops and webinars; and (3) consultations with specific stakeholder groups, Consortium members and a consultative body formed by international PE initiatives. Results This roadmap sets strategic goals for the PE community to achieve meaningful and systematic PE through changes in the culture, processes and resources of stakeholder organisations. It brings in key PARADIGM outputs to work in a coordinated fashion with existing frameworks and mechanisms to achieve system-wide sustained PE. Conclusions The roadmap provides a framework for all stakeholders to take collective action within their organisations and across Europe to implement PE in a sustainable manner.

Background The holistic evolution of patient engagement in medicines development requires a more detailed understanding of the needs of all involved stakeholders, and one that better accounts for the specific needs of some potentially vulnerable patient populations and key stages in medicines development. Objective The purpose of this convergent mixed‐methods study was to better understand the needs of different stakeholders concerning patient engagement at three key stages in medicines development: research priority setting, clinical trial design and early dialogues with Health Technology Assessment bodies and regulators. Design This study brought together findings from three sources: i) an online questionnaire, ii) face‐to‐face consultations with two potentially vulnerable patient populations, a workshop with Health Technology Assessment bodies, and iii) three‐step modified Delphi methodology. Results Overall stakeholders still need additional varied support mechanisms to undertake, sustain or measure value of patient engagement. Health Technology Assessment bodies need better rationale for patient engagement in early dialogue and tools to support its implementation. Improved awareness and understanding of the need and value that involving patients, who are often considered as potentially vulnerable, can bring is needed, as is better accommodation of their specific needs. Similarly, weighted Delphi categories were as follows: aims and objectives, and sustainability. Several additional themes were common across the three key stages in medicines development. Conclusion This broad‐reaching study provides the blocks needed to build a framework for patient engagement in medicines development. Patient or Public Contribution Patients were involved in review and interpretation of data.