The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments

Langfeldt, Liv

doi:10.3152/147154406781776039

Cited by 130 publications

(106 citation statements)

References 0 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…Importantly, publication-based metrics provide an objective counterweight in tenure and promotion discussions to the peer-review process, which is prone to bias of many kinds 10 .…”

Section: David Pendleburymentioning

confidence: 99%

How to improve the use of metrics

2010

Nature

View full text Add to dashboard Cite

show abstract

“…Importantly, publication-based metrics provide an objective counterweight in tenure and promotion discussions to the peer-review process, which is prone to bias of many kinds 10 .…”

Section: David Pendleburymentioning

confidence: 99%

How to improve the use of metrics

2010

Nature

View full text Add to dashboard Cite

show abstract

“…One suggestion is that different reviewers with different cognitive biases should be used for different schemes -specifically targeting specialists in translational or high-risk, innovative research where this is the desired outcome -and that there could be more flexibility around qualities such as stringency and degrees of selectivity, which should be adjusted to the objectives of the review (Langfeldt 2006). This is an approach which has been used in some of NIH's high-risk, high-reward programmes, such as the Pioneer awards, which go through a two-panel review process, with the first consisting of three generalist reviewers who have a broad view of science and are not allowed to discuss the applications with each other.…”

Section: Alternative Approaches That May Help Foster Innovative Researchmentioning

confidence: 99%

What do we know about grant peer review in the health sciences? An updated review of the literature and six case studies

Guthrie¹,

Ghiga²,

Wooding³

2018

View full text Add to dashboard Cite

An updated review of the literature and six case studies RAND Europe is a not-for-profit organisation whose mission is to help improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors. Limited Print and Electronic Distribution RightsThis document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. Support RANDMake a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org www.rand.org/randeurope For more information on this publication, visit www.rand.org/t/RR1822Published by the RAND Corporation, Santa Monica, Calif., and Cambridge, UK © Copyright 2018 RAND CorporationR® is a registered trademark.ii PrefaceIn 2009, RAND Europe conducted a literature review in order to assess the effectiveness and efficiency of peer review for grant funding. This report presents an update to that review to reflect new literature on the topic, and adds case studies exploring peer review practice at six international funders.This report was produced with funding from the Canadian Institutes of Health Research. It will be of interest to government officials dealing with research funding policy, research funders including governmental and charitable funders, research institutions, researchers, and research users. Although the case studies focus on biomedical and health research, the literature review takes a broader scope and it is likely the findings may be of relevance to wider research fields.For further information on this publication please contact: SummaryThough often viewed as the 'gold standard' process of quality assurance for research, grant peer review has also received significant criticism from both within and outside academia. Detractors highlight inefficiency and structural flaws that compromise its effectiveness in allocating funding. In 2009 we conducted a review of the literature to evaluate these criticisms. 1 This new report updates that literature review, and also provides case studies of current practice across six major international biomedical and health research funders. The work was commissioned by the Canadian Institutes of Health Research (CIHR) with the aim of supporting the ongoing review of their peer review system, particularly the forthcoming work of the Peer Review Expert Panel convened to review the design and adjudication processes of CIHR's investigator-initiated research programmes. What is grant peer review?In its most basic form, peer review involves academic reviewers in the process...

show abstract

“…Autores como Laudel (2006) consideran que es posible mantener criterios tradicionales de evaluación frente a autores como Klein (2008), que creen que un método único puede distorsionar la multidimensionalidad de la propia investigación. Langfeldt (2006), por su parte, señala la importancia de la evaluación tradicional siempre que pueda ser complementada cuando y dónde sea necesario.…”

Section: Conclusiones Y Discusiónunclassified

“…Klein (2008), por su parte, opina que no se deberían imponer metodologías ni impedir la evaluación a la espera de un método único y universal de medición que sería la antítesis de la multidimensionalidad de la propia investigación. Langfeldt (2006), en una postura intermedia, señala que es importante que la selección de revisores sea adecuada y se eviten confl ictos, que se tenga claro a priori qué tipo de investigación se va a promover (convencional o interdisciplinar), y que se complemente la evaluación por pares con otros comités cuando sea preciso.…”

Section: Introducción Y Objetivosunclassified

Características y disparidades entre sub-especialidades: un estudio de caso con grupos de Biotecnología

Morillo¹,

Aparicio²

2011

Rev. esp. doc. cient.

View full text Add to dashboard Cite

Resumen: El objetivo de este trabajo es el estudio de las características y disparidades entre las sub-especialidades existentes dentro de categorías interdisciplinares, planteando la necesidad de identifi car y evaluar por separado cada una de ellas. Se analizan los años 2003-2006, a través de las bases de datos del Web of Science (WoS), y se exploran las posibles diferencias entre grupos de investigación de Biotecnología de la Comunidad de Madrid según su perfi l de especialización. Esta región española es la de mayor producción en Biotecnología (27 %) y dicha categoría presenta interacciones con otras disciplinas como Microbiología, Bioquímica y Alimentos. Por este motivo, el estudio de grupos incluye, en un segundo análisis, no sólo los documentos de esta categoría, sino también todas las publicaciones de los investigadores principales, independientemente de su disciplina. Con estos datos, se examina la infl uencia de la especialización en las características de los grupos, sus patrones de colaboración y su impacto. Se identifi can tres tipologías según las sub-especialidades: a) grupos del NÚ-CLEO (publicando el 50 % o más de sus documentos en Biotecnología); b) grupos de tipo BÁSICO (publicando en Bioquímica y otras disciplinas básicas), y c) grupos de tipo MEDIO (publicando en Alimentos y disciplinas menos básicas). Por medio de pruebas estadísticas, se encontraron diferencias signifi cativas entre estos conjuntos en relación con su producción e impacto. Se concluye que estas disparidades deberían ser tenidas en cuenta al evaluar la investigación que se produce en categorías interdisciplinares como la Biotecnología.Palabras clave: Indicadores bibliométricos, sub-especialidades, evaluación, grupos de investigación, Biotecnología. Characteristics and disparities among sub-specialities: a case study of Biotechnology research teams

show abstract

The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments

Cited by 130 publications

References 0 publications

How to improve the use of metrics

How to improve the use of metrics

What do we know about grant peer review in the health sciences? An updated review of the literature and six case studies

Características y disparidades entre sub-especialidades: un estudio de caso con grupos de Biotecnología

Contact Info

Product

Resources

About