Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits

Wallace, Matthew L.; Ràfols, Ismael

doi:10.1007/s11024-015-9271-8

Cited by 60 publications

(34 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Funding bodies allocate resources among topics, administrators choose which researchers to hire and which projects to support internally, while researchers (for the most part) choose the topics they want to work on (Fisher, 2005;Foster, Rzhetsky, & Evans, 2015;Zuckerman, 1978). The notions of research portfolios and portfolio analysis, once largely confined to the corporate R&D world, are now being increasingly considered in academic and agency settings (Wallace & Rafols, 2015).…”

Section: Introductionmentioning

confidence: 99%

Research portfolio analysis and topic prominence

Klavans

Boyack

2017

Journal of Informetrics

View full text Add to dashboard Cite

Stakeholders in the science system need to decide where to place their bets. Example questions include: Which areas of research should get more funding? Who should we hire? Which projects should we abandon and which new projects should we start? Making informed choices requires knowledge about these research options. Unfortunately, to date research portfolio options have not been defined in a consistent, transparent and relevant manner. Furthermore, we don't know how to define demand for these options. In this article, we address the issues of consistency, transparency, relevance and demand by using a model of science consisting of 91,726 topics (or research options) that contain over 58 million documents. We present a new indicator of topic prominence -a measure of visibility, momentum and, ultimately, demand. We assign over $203 billion of project-level funding data from STAR METRICS ® to individual topics in science, and show that the indicator of topic prominence, explains over one-third of the variance in current (or future) funding by topic. We also show that highly prominent topics receive far more funding per researcher than topics that are not prominent. Implications of these results for research planning and portfolio analysis by institutions and researchers are emphasized.

show abstract

Section: Introductionmentioning

confidence: 99%

Research portfolio analysis and topic prominence

Klavans

Boyack

2017

Journal of Informetrics

View full text Add to dashboard Cite

show abstract

“…371 Both HEDIIP and HEIDI are focused on the teaching side of the higher education system, where data has in large part been managed separately from the research side, but with discussions now underway at a policy level about the introduction of a 'teaching excellence framework' (which may include its own outcome-focused metrics), 372 the scope for sensitive integration of processes across the system as a whole is likely to increase. Some countries are beginning to draw on scientometric mapping and analysis in more creative and strategic ways to shape technology and innovation policies, 374 to determine funding priorities and manage portfolios, 375 to measure the wider impacts of research, 376 or to support wider approaches to foresight and horizon scanning. 377 The UK has a 30-year history of research assessment, and there is good and varied evidence that the overall quality of UK research has improved over that period.…”

Section: Smarter Science and Innovation Policymentioning

confidence: 99%

Approaching Impact Meaningfully in Medical Education Research

et al. 2019

View full text Add to dashboard Cite

Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. If we as a sector can't take full advantage of the possibilities of big data, then who can? Yet we only have to look around us, at the blunt use of metrics such as journal impact factors, h-indices and grant income targets to be reminded of the pitfalls. Some of the most precious qualities of academic culture resist simple quantification, and individual indicators can struggle to do justice to the richness and plurality of our research. Too often, poorly designed evaluation criteria are "dominating minds, distorting behaviour and determining careers." 1 At their worst, metrics can contribute to what Rowan Williams, the former Archbishop of Canterbury, calls a "new barbarity" in our universities. 2 The tragic case of Stefan Grimm, whose suicide in September 2014 led Imperial College to launch a review of its use of performance metrics, is a jolting reminder that what's at stake in these debates is more than just the design of effective management systems. 3 Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to positive ends is the focus of this report. Based on fifteen months of evidence-gathering, analysis and consultation, we propose here a framework for responsible metrics, and make a series of targeted recommendations. Together these are designed to ensure that indicators and underlying data infrastructure develop in ways that support the diverse qualities and impacts of UK research. Looking to the future, we show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework. The metric tide is certainly rising. Unlike King Canute, we have the agency and opportunityand in this report, a serious body of evidenceto influence how it washes through higher education and research. Let me end on a note of personal thanks to my steering group colleagues, to the team at HEFCE, and to all those across the community who have contributed to our deliberations.

show abstract

“…The emergence of these new concepts and their career in the 1970s and 1980s is of interest here because in the recent STI policy literature, the GC discourse is often conceived of as a reformulation of the idea of mission-oriented research (Gassler et al 2008 ; Cagnin et al 2012 ; Foray et al 2012 ; Amanatidou et al 2014 ; Wallace and Rafols 2015 ). Furthermore, when discussing the grand challenges of our time, many authors, including actors in the field of science policy and scholars producing the secondary literature, point to the legendary 20th-century “mission oriented” or “big science” research endeavors.…”

Section: Part I: From Problems To Challengesmentioning

confidence: 99%

The Grand Challenges Discourse: Transforming Identity Work in Science and Science Policy

Kaldewey

2017

Minerva

View full text Add to dashboard Cite

This article analyzes the concept of “grand challenges” as part of a shift in how scientists and policymakers frame and communicate their respective agendas. The history of the grand challenges discourse helps to understand how identity work in science and science policy has been transformed in recent decades. Furthermore, the question is raised whether this discourse is only an indicator, or also a factor in this transformation. Building on conceptual history and historical semantics, the two parts of the article reconstruct two discursive shifts. First, the observation that in scientific communication references to “problems” are increasingly substituted by references to “challenges” indicates a broader cultural trend of how attitudes towards what is problematic have shifted in the last decades. Second, as the grand challenges discourse is rooted in the sphere of sports and competition, it introduces a specific new set of societal values and practices into the spheres of science and technology. The article concludes that this process can be characterized as the sportification of science, which contributes to self-mobilization and, ultimately, to self-optimization of the participating scientists, engineers, and policymakers.Electronic supplementary materialThe online version of this article (doi:10.1007/s11024-017-9332-2) contains supplementary material, which is available to authorized users.

show abstract

Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits

Cited by 60 publications

References 86 publications

Research portfolio analysis and topic prominence

Research portfolio analysis and topic prominence

Approaching Impact Meaningfully in Medical Education Research

The Grand Challenges Discourse: Transforming Identity Work in Science and Science Policy

Contact Info

Product

Resources

About