Evaluating health research impact: Development and implementation of the Alberta Innovates - Health Solutions impact framework

Graham, Kathryn E.; Chorzempa, Heidi L.; Valentine, Pamela A.; Magnan, Jacques

doi:10.1093/reseval/rvs027

Cited by 41 publications

(51 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, in addition to continuing refinement of existing frameworks, for example, of the Canadian Academy of Health Sciences’ framework in Canada [87], there are also ever-increasing numbers of studies on which to draw to inform analysis, including current work in Australia [88]. Given the expanding focus on research impact assessment, the potential lessons that could be drawn from them, individually and collectively, are likely to be more significant if there could be somewhat greater standardisation.…”

Section: Discussionmentioning

confidence: 99%

The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews

Hanney

Greenhalgh

Blatch-Jones

et al. 2017

Health Res Policy Sys

View full text Add to dashboard Cite

Background: We sought to analyse the impacts found, and the methods used, in a series of assessments of programmes and portfolios of health research consisting of multiple projects. Methods: We analysed a sample of 36 impact studies of multi-project research programmes, selected from a wider sample of impact studies included in two narrative systematic reviews published in 2007 and 2016. We included impact studies in which the individual projects in a programme had been assessed for wider impact, especially on policy or practice, and where findings had been described in such a way that allowed them to be collated and compared.

show abstract

Section: Discussionmentioning

confidence: 99%

The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews

Hanney

Greenhalgh

Blatch-Jones

et al. 2017

Health Res Policy Sys

View full text Add to dashboard Cite

show abstract

“…It was developed to improve the organisation's ability to assess its contributions to health systems impacts, in addition to the contributions of its grantees. 85 The CAHS framework has also been applied in Catalonia by the Catalan Agency for Health Information and Quality. 84 Banzi's research impact model Banzi et al,4 in a review of the literature on research impact assessment, identified the Payback Framework as the most frequently used approach.…”

Section: Updated Systematic Reviewmentioning

confidence: 99%

“…85 The scale of application varied enormously, with some frameworks using small-scale approaches designed for use by individual researchers, such as the RIF developed by Kuruvilla et al 97 At the other end of the spectrum national schemes continue in the Netherlands, 79 and have developed with the UK's REF. 106 This reflects the growing interest by research funders from national governments through to small funding organisations.…”

Section: Updated Systematic Reviewmentioning

confidence: 99%

Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme

Raftery

Hanney

Greenhalgh

et al. 2016

Health Technol Assess

View full text Add to dashboard Cite

BackgroundThis report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.Objectives(1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.Data sourcesWe searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.Review methodsThis narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.ResultsThe literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.DiscussionThe findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence’s remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities’ research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish®(researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.LimitationsThere were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.ConclusionsResearch funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines.FundingThe National Institute for Health Research HTA programme.

show abstract

“…Research ontologies focus on the researchers, the research products and procedures. CASRAI (The Consortia Advancing Standards in Research Administration Information) standards refer to the terminology of the semantics and the structure of research information [17], the life-cycle of research and the research impact [18]. CERIF (Common European Research Information Format) corresponds to a canonical reference data model for data and metadata about research objects and their relationships [19,20].…”

Section: Modeling Academic Activitymentioning

confidence: 99%

Creating the Context for Exploiting Linked Open Data in Multidimensional Academic Ranking

Triperina

Sgouropoulou

Xydas

et al. 2015

Int. J. Recent Contrib. Eng. Sci. IT

View full text Add to dashboard Cite

Abstract-Academia is a complex socio-technical system with multiple aspects and constituents that involve various stakeholders. In order to address stakeholders' needs and to assist the institutional accountability, this complexity should be considered during the development of academic services. We have designed a dynamic multidimensional ranking approach, easily modifiable to address user requirements, so as to assess and compare the university performance with a clear view to the support of effective institutional strategic planning and policy making. Our approach comprises the following components: the AcademIS ontology to model the academic domain and its multiple dimensions, the AcademIS Information System to manage and display the academic information, published in Linked Open Data format and the visual-aided Multiple Criteria Decision Making component, to evaluate and rank the performance of the academic units. The data are aggregated from several sources, in different formats, LODified by our system, and presented to the user by the interface to ultimately assist the decision making process.

show abstract

Evaluating health research impact: Development and implementation of the Alberta Innovates - Health Solutions impact framework

Cited by 41 publications

References 7 publications

The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews

The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews

Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme

Creating the Context for Exploiting Linked Open Data in Multidimensional Academic Ranking

Contact Info

Product

Resources

About