What funders are doing to assess the impact of their investments in health and biomedical research

Abstract

As pressures to maximize research funding grow, biomedical research funders are increasingly tasked with demonstrating the long-term and real-world impacts of their funded research investments. Over the past three decades, research impact assessments (RIA) have emerged as an important tool for analysing the impacts of research by incorporating logic models, frameworks and indicators to track measures of knowledge production, capacity-building, development of research products, adoption of research into clinical guidelines and policies, and the realization of health, economic and social benefits. While there are currently several models for RIA within the literature, less attention has been paid to how funders can practically select and implement a RIA model to demonstrate the impacts of their own research portfolios. In this paper, a literature review was performed to understand (1) which research funders have performed RIAs of their research portfolios to date; (2) how funders have designed their assessments, including the models and tools they have used; (3) what challenges to and facilitators of success have funders found when adopting the RIA model to their own portfolio; and (4) who participates in the assessments. Forty-four papers from both published and grey literature were found to meet the review criteria and were examined in detail. There is a growing culture of RIA among funders, and included papers spanned a diverse set of funders from 10 countries or regions. Over half of funders (59.1%) used a framework to conduct their assessment, and a variety of methods for collecting impact data were reported. Issues of methodological rigour were observed across studies in the review, and this was related to numerous challenges funders faced in designing timely RIAs with quality impact data. Over a third of articles (36.4%) included input from stakeholders, yet only one article reported surveying patients and members of the public as part of the assessment. To advance RIA among funders, we offer several recommendations for increasing the methodological rigour of RIAs and suggestions for future research, and call for a careful reflection of the voices needed in an impact assessment to ensure that RIAs are having a meaningful impact on patients and the public.

How to strengthen societal impact of research and innovation? Lessons learned from assessing participatory knowledge infrastructures funded by the Netherlands Organization for Health Research and Development

“…Several frameworks have been developed (e.g., payback model, (hybrid) Research Impact Framework, health economics models, contribution mapping) [4,5,6,7,8] and several studies have been conducted to assess the societal impact of research and innovation, e.g., [9,10,11]. From these studies it is known that successful knowledge utilisation involves so-called productive interactions [12].…”

“…Productive interactions have increasingly gained interest of (health) funders around the globe [4,6]. One of these funders, the Netherlands Organization for Health Research and Development (ZonMw), funds and promotes (the use of) health research and care innovation throughout the entire knowledge chain from fundamental research to implementation projects.…”

A case study of charitable investment impact assessment

“…The literature argues persuasively that biomedical research funders, such as foundations like Pablove, are increasingly driven to demonstrate research impact assessments (RIA). Nonetheless, a study by Abudu et al, 2022 only surfaced 44 organizations (worldwide) that documented RIA in the literature and only 59.1% followed a specific conceptual framework. The remaining funders followed no framework.…”

“…And some argue that precious resources should not be invested in "over-assessing" performance because intuitively results are apparent in the day-to-day activities and operations, amount of funds raised and allocated, stunning digital images created by the Shutterbugs, and active participation by new clients.The literature argues persuasively that biomedical research funders, such as foundations like Pablove, are increasingly driven to demonstrate research impact assessments (RIA). Nonetheless, a study byAbudu et al, 2022 only surfaced 44 organizations (worldwide) that documented RIA in the literature and only 59.1% followed a specific conceptual framework. The remaining funders followed no framework.…”

Outcomes of an integrated knowledge translation approach in five African countries: A mixed methods comparative case study

“…Interest in IKT has increased, including among research funders who view the involvement of knowledge users and KT activities as critical for achieving research impact (9,10), which serves as a justi cation for public funding of research (11).…”