Preparing impact submissions for REF 2014: An evaluation: Findings and observations

We Need to Talk about Impact: Why Social Policy Academics need to Engage with the UK's Research Impact Agenda

“…Black, 2001; Lavis et al , 2004). Several examples of systematic reviews being cited in policy debates were provided by interviewees and, in these cases, interviewees seemed to suggest that credit for impact ought to be attributed to the author(s) of the review(s) (a finding which seems in line with REF2014 guidance but which contrasts somewhat with the findings of Manville et al ’s 2014 evaluation, in which some participants emphasised the importance of acknowledging the key studies cited within reviews). However, none of our interviewees said that they had yet developed an impact case study around a systematic review and one interviewee reported being unable to persuade a senior colleague involved in collating his (social policy) department's REF submission (for the social work and social policy panel) that a systematic review constituted a valid piece of research.…”

“…Reflecting this, around one-third of the academics interviewed suggested that the UK's ‘impact agenda’ could inadvertently (or even, a few suggested, deliberately) encourage researchers to pursue work that is sympathetic to existing, short-term policy directions, on the basis that such research is more likely to have a traceable policy impact; to, as one interviewee put it, ‘bend with the wind in order to get research cited’. Indeed, several academics openly reflected that the current emphasis on producing policy-relevant research was leading them to limit the critical aspects of their work, at least in non-academic contexts (a similar concern regarding the potential threat to ‘blue skies’ work is evident in Manville et al ’s 2014 evaluation). These findings reflect broader concerns that efforts to achieve evidence-based policy may, in fact, do more to promote policy-informed research (e.g.…”

“…However, as the quotations above indicate, several interviewees suggested that they were not convinced that the quality threshold for 'impact' was sufficiently high. In this context, it is worth noting that an evaluation of the REF2014 impact assessment, commissioned by the UK higher education bodies and undertaken by RAND (Manville et al, 2014), identified concerns among some assessors about their ability to assess whether the research underlying impact case studies met the 2 * threshold. Similarly, one interviewee described feeling that an advocacy group had deliberately reframed the implications of their research, despite their protestations (see also Dagnino, 2007).…”

Impact case studies submitted to REF 2014: The hidden impact of nursing research

“…The inclusion of an assessment of the impact of research remains a relatively new process, the overall aim being to demonstrate the societal benefit of research. This is a laudable but complex goal and an evaluation study by Rand Europe was conducted into the impact cases of REF 2014 (Manville et al., 2015a, 2015b). Following an in-depth analysis of all submitted impact cases, they offered three conclusions: first, that the HEFCE impact case repository offers a rich source of research material (hence this study); second, that the range and diversity of impact cases would suggest that identification of a common metric for judging impact would be unlikely to succeed; and third, that some common nomenclature and definitions around impact would be helpful for future exercises.…”

“…Since the 2014 results were announced, a number of authors have analysed the findings, the most comprehensive study having been carried out by the Policy Institute at King's College London (Manville et al., 2015a, 2015b). In addition there have been published analyses of specific disciplines such as one by Greenhalgh and Fahy (2015), who argue against the dominant linear and short-term nature of many of the impact cases submitted to sub-panel UoA2.…”