Money for nothing and your referees for free

Lortie, Christopher J.

doi:10.4033/iee.2011.4.9.e

Cited by 10 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, one could argue that reputed referees should not be substituted by non-experts. However, it is a fact that the growth in the number of submissions threatens the sustainability of a system that leans on voluntary cooperation (Lortie 2011 ; Kovanis et al. 2016b ).…”

Section: Introductionmentioning

confidence: 99%

Reputation or peer review? The role of outliers

2018

View full text Add to dashboard Cite

We present an agent-based model of paper publication and consumption that allows to study the effect of two different evaluation mechanisms, peer review and reputation, on the quality of the manuscripts accessed by a scientific community. The model was empirically calibrated on two data sets, mono- and multi-disciplinary. Our results point out that disciplinary settings differ in the rapidity with which they deal with extreme events—papers that have an extremely high quality, that we call outliers. In the mono-disciplinary case, reputation is better than traditional peer review to optimize the quality of papers read by researchers. In the multi-disciplinary case, if the quality landscape is relatively flat, a reputation system also performs better. In the presence of outliers, peer review is more effective. Our simulation suggests that a reputation system could perform better than peer review as a scientific information filter for quality except when research is multi-disciplinary and in a field where outliers exist.Electronic supplementary materialThe online version of this article (10.1007/s11192-018-2826-3) contains supplementary material, which is available to authorized users.

show abstract

Section: Introductionmentioning

confidence: 99%

Reputation or peer review? The role of outliers

2018

View full text Add to dashboard Cite

show abstract

“…Collectively, articles are the primary source of vetted and reputable scientific information (Bornmann 2011). Although the peer-review process (Weber et al 2002;Smith 2006;Resnik 2011) and publication enterprise (Lortie 2011) are not without criticism, such articles remain the bread and butter of contemporary knowledge sharing within the scholarly community (Kassirer and Campion 1994). The substance of journal articles (hereinafter, paper(s)) drives the scientific endeavour (Robertson 2009;Bornmann 2011).…”

Section: Introductionmentioning

confidence: 99%

Envisioning the scientific paper of the future

Sopinka¹,

Coristine

DeRosa

et al. 2020

FACETS

View full text Add to dashboard Cite

Consider for a moment the rate of advancement in the scientific understanding of DNA. It is formidable; from Fredrich Miescher’s nuclein extraction in the 1860s to Rosalind Franklin’s double helix X-ray in the 1950s to revolutionary next-generation sequencing in the late 2000s. Now consider the scientific paper, the medium used to describe and publish these advances. How is the scientific paper advancing to meet the needs of those who generate and use scientific information? We review four essential qualities for the scientific paper of the future: ( i) a robust source of trustworthy information that remains peer reviewed and is ( ii) communicated to diverse users in diverse ways, ( iii) open access, and ( iv) has a measurable impact beyond Impact Factor. Since its inception, scientific literature has proliferated. We discuss the continuation and expansion of practices already in place including: freely accessible data and analytical code, living research and reviews, changes to peer review to improve representation of under-represented groups, plain language summaries, preprint servers, evidence-informed decision-making, and altmetrics.

show abstract

“…This is probably because they involve complicated systems of payment or non-financial incentives (e.g. [7][8][9]), or penalize reviewers' poor performance instead of rewarding their efforts (e.g. [10]).…”

Section: Introductionmentioning

confidence: 99%

The missing metric: quantifying contributions of reviewers

Cantor

Gero

2015

R. Soc. open sci.

View full text Add to dashboard Cite

The number of contributing reviewers often outnumbers the authors of publications. This has led to apathy towards reviewing and the conclusion that the peer-review system is broken. Given the trade-offs between submitting and reviewing manuscripts, reviewers and authors naturally want visibility for their efforts. While study after study has called for revolutionizing publication practices, the current paradigm does not recognize reviewers' time and expertise. We propose the R-index as a simple way to quantify scientists' contributions as reviewers. We modelled its performance using simulations based on real data to show that early–mid career scientists, who complete high-quality reviews of longer manuscripts within their field, can perform as well as leading scientists reviewing only for high-impact journals. By giving citeable academic recognition for reviewing, R-index will encourage more participation with better reviews, regardless of the career stage. Moreover, the R-index will allow editors to exploit scores to manage and improve their review team, and for journals to promote high average scores as signals of a practical and efficient service to authors. Peer-review is a pervasive necessity across disciplines and the simple utility of this missing metric will credit a valuable aspect of academic productivity without having to revolutionize the current peer-review system.

show abstract

Money for nothing and your referees for free

Cited by 10 publications

References 9 publications

Reputation or peer review? The role of outliers

Reputation or peer review? The role of outliers

Envisioning the scientific paper of the future

The missing metric: quantifying contributions of reviewers

Contact Info

Product

Resources

About