Conservatism and risk-taking in peer review: Emerging ERC practices

Luukkonen, Terttu

doi:10.1093/reseval/rvs001

Cited by 155 publications

(139 citation statements)

References 9 publications

Supporting

Mentioning

136

Contrasting

Order By: Relevance

“…Some have suggested that highly innovative research may be associated with more reviewer uncertainty about their judgments of the methodology, which may lead to lower scores (Luukkonen 2012;Boudreau et al 2016). While reviewers may consider innovation in their decisions, they likely give this less weight than methodological weaknesses, which is reflected in the content of the reviewer's critique.…”

Section: Attitudes Toward Innovation and Riskmentioning

confidence: 99%

Risk evaluation in peer review of grant applications

et al. 2018

View full text Add to dashboard Cite

The process of peer review is used to identify the most scientifically meritorious research projects for funding. Impact and innovation are among the criteria used to determine overall merit. A criticism of peer review has been the perception that reviewers are biased against innovation, such as one study that found reviewers to systematically assign poorer scores to highly novel work. Moreover, reviewers' definitions for excellent research and paradigm-shifting research are different; innovative research may not always be considered excellent. Therefore, it is clear more needs to be done to understand the decision-making processes of reviewers in evaluating risk and innovation in research. In an effort to address this gap, the American Institute of Biological Sciences developed a comprehensive peer review survey that examined, in part, the differences in applicant and reviewer perceptions of review outcomes. The survey was disseminated to 13,091 reviewers and applicants, of whom 9.4% responded. Only 24% of respondent applicants indicated that innovation was addressed in their review feedback, while 81% of respondent reviewers indicated they factored innovation into selecting the best science and 73% viewed innovation as an essential component of scientific excellence. Similarly, while only 27% of respondent applicants reported receiving comments on the riskiness of their grant applications, 58% of respondent reviewers indicated that the risks associated with innovative research impacted the scores they assigned to the grant applications. These results indicate a potential source of bias in how innovation and risk are evaluated in grant applications.

show abstract

Section: Attitudes Toward Innovation and Riskmentioning

confidence: 99%

Risk evaluation in peer review of grant applications

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Scholarly interest in funding agencies ceased before interesting variations in their relative autonomy from the state and scientific communities (Braun 1998) or field-specific effects on the behaviour of researchers (Morris 2000) were fully investigated and theorised. Attention shifted to the study of funding programmes (Aguilar et al 1998;Melin and Danell 2006;Heinze 2008;Edler et al 2014), and here mostly to selection processes (Bornmann and Daniel 2005;Lamont 2009;Luukkonen 2012). This move has certainly been promoted by the growing interest of funders in the efficacy of their funding schemes and the commissioning of evaluation reports (Hornbostel et al 2009;Thomas and Nedeva 2012;Möller et al 2016).…”

Section: Targeting Changes In the Funding Of Sciencementioning

confidence: 99%

Changing Funding Arrangements and the Production of Scientific Knowledge: Introduction to the Special Issue

Gläser

Velarde

2018

Minerva

View full text Add to dashboard Cite

With this special issue, we would like to promote research on changes in the funding of the sciences, social sciences, and humanities. Since funding secures the livelihood of researchers and the means to do research, it is an indispensable condition for almost all research; as funding arrangements are undergoing dramatic changes, we think it timely to renew the science studies community's efforts to understand the funding of research. Changes in the governance of science have garnered considerable attention from science studies and higher education research; however, the impact of these changes on the conduct and content of research has not received sufficient attention, and theoretical insights into the connections between funding practices and research practices are few and far between. The aim of this special issue is to contribute to our theoretical understanding of the changing nature of research funding and its impact on the production of scientific knowledge. More specifically, we are interested in the interplay between funding and research practices: What is the impact of institutionalised funding arrangements on the production of scientific knowledge?

show abstract

“…Including innovation specifically as one of the assessment criteria is another approach, aiming to actively change the way reviewers consider proposals (Lindner et al 2016;Luukkonen 2012). Views of the use of innovation as an assessment criterion have been mixed, with some suggesting that panels are not well placed to assess innovation because the expertise required is often lacking (Costello 2010), whilst others suggest that the approach is effective in rewarding innovation (Spiegel 2010).…”

Section: Alternative Approaches That May Help Foster Innovative Researchmentioning

confidence: 99%

What do we know about grant peer review in the health sciences? An updated review of the literature and six case studies

Guthrie¹,

Ghiga²,

Wooding³

2018

View full text Add to dashboard Cite

An updated review of the literature and six case studies RAND Europe is a not-for-profit organisation whose mission is to help improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors. Limited Print and Electronic Distribution RightsThis document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. Support RANDMake a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org www.rand.org/randeurope For more information on this publication, visit www.rand.org/t/RR1822Published by the RAND Corporation, Santa Monica, Calif., and Cambridge, UK © Copyright 2018 RAND CorporationR® is a registered trademark.ii PrefaceIn 2009, RAND Europe conducted a literature review in order to assess the effectiveness and efficiency of peer review for grant funding. This report presents an update to that review to reflect new literature on the topic, and adds case studies exploring peer review practice at six international funders.This report was produced with funding from the Canadian Institutes of Health Research. It will be of interest to government officials dealing with research funding policy, research funders including governmental and charitable funders, research institutions, researchers, and research users. Although the case studies focus on biomedical and health research, the literature review takes a broader scope and it is likely the findings may be of relevance to wider research fields.For further information on this publication please contact: SummaryThough often viewed as the 'gold standard' process of quality assurance for research, grant peer review has also received significant criticism from both within and outside academia. Detractors highlight inefficiency and structural flaws that compromise its effectiveness in allocating funding. In 2009 we conducted a review of the literature to evaluate these criticisms. 1 This new report updates that literature review, and also provides case studies of current practice across six major international biomedical and health research funders. The work was commissioned by the Canadian Institutes of Health Research (CIHR) with the aim of supporting the ongoing review of their peer review system, particularly the forthcoming work of the Peer Review Expert Panel convened to review the design and adjudication processes of CIHR's investigator-initiated research programmes. What is grant peer review?In its most basic form, peer review involves academic reviewers in the process...

show abstract

Conservatism and risk-taking in peer review: Emerging ERC practices

Cited by 155 publications

References 9 publications

Risk evaluation in peer review of grant applications

Risk evaluation in peer review of grant applications

Changing Funding Arrangements and the Production of Scientific Knowledge: Introduction to the Special Issue

What do we know about grant peer review in the health sciences? An updated review of the literature and six case studies

Contact Info

Product

Resources

About