Research impact: a narrative review

Abstract

Impact occurs when research generates benefits (health, economic, cultural) in addition to building the academic knowledge base. Its mechanisms are complex and reflect the multiple ways in which knowledge is generated and utilised. Much progress has been made in measuring both the outcomes of research and the processes and activities through which these are achieved, though the measurement of impact is not without its critics. We review the strengths and limitations of six established approaches (Payback, Research Impact Framework, Canadian Academy of Health Sciences, monetisation, societal impact assessment, UK Research Excellence Framework) plus recently developed and largely untested ones (including metrics and electronic databases). We conclude that (1) different approaches to impact assessment are appropriate in different circumstances; (2) the most robust and sophisticated approaches are labour-intensive and not always feasible or affordable; (3) whilst most metrics tend to capture direct and proximate impacts, more indirect and diffuse elements of the research-impact link can and should be measured; and (4) research on research impact is a rapidly developing field with new methodologies on the horizon.

Assessing (for) impact: future assessment of the societal impact of research

“…There are a number of well-established challenges in assessing the impact from research, principally the interrelated concerns of the sometimes long time lags between research and impact, the problems of attribution of research to impact and the difficulties in providing evidence of the links between research and impact (Bornmann, 2013;Penfield et al, 2014;Greenhalgh et al, 2016). These issues are further complicated by the complex, and potentially non-linear and non-sequential relationship between research and impact.…”

Disseminação e uso dos resultados de pesquisas financiadas pela Secretaria de Ciência, Tecnologia e Insumos Estratégicos do Ministério da Saúde, Brasil, 2004 a 2007

“…As categorias de uso do modelo são: conhecimento (por exemplo, publicações acadêmicas), benefícios para futuras pesquisas (por exemplo, formação de novos pesquisadores), benefícios para a política (por exemplo, base de informação para tomadores de decisão), benefícios para a saúde da população ou para o sistema de saúde (por exemplo, indicadores de morbimortalidade, redução de custos, qualidade das ações) e benefícios econômicos mais amplos (por exemplo, ampliação da cadeia produtiva na saúde) (HANNEY ET AL., 2003, 2004GREENHALGH ET AL., 2016).…”

“…Por isso, uma das abordagens para avaliar o impacto de uma pesquisa poderá combinar um modelo lógico para descrever ligações entre objetivos, atividades e produtos, com o estudo de caso para verificar as interações complexas entre os atores e as instituições, através do qual o conhecimento é produzido e compartilhado (GREENHALGH ET AL., 2016).…”

Learning from the UK’s research impact assessment exercise: a case study of a retrospective impact assessment exercise and questions for the future

“…(Spaapen et al 2013, p.4) The highlighted words in the various definitions above point out a central and common the idea that what is being sought is a diverse set of tangible benefits for society and the economy. However, this is not to say that there is, necessarily, agreement on the philosophical underpinnings, nor definition of impact (Penfield et al 2014;Greenhalgh et al 2016). While we do not intend to resolve that debate here, we do aim to point out that there is a common thread running through different definitions, and there are a corresponding set of methods and frameworks that have evolved to try and help researchers and policymakers understand impact.…”

“…With the growth of the field of research impact evaluation in recent years, newer methods for assessing impact are now beginning to also emerge, including forms of data mining and analysis of multiple, alternative metrics, for understanding how and by whom research is being used (Priem et al 2012). Each of these approaches and methods can be designed and used to highlight particular societal impacts (Guthrie et al 2013;Greenhalgh et al 2016). However, under any circumstances, demonstrating, measuring, evaluating and understanding impact is not straightforward and there are different methodological drivers and corresponding challenges of research impact assessments.…”