A multi-disciplinary perspective on emergent and future innovations in peer review

Abstract

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Preprints in Scholarly Communication: Re-Imagining Metrics and Infrastructures

“…The peer review process exists to enable nominally disinterested experts to assure the quality of academic publications, but preprint servers usually host articles that have not yet been subject to peer review. The question of peer review at this juncture is-for open science-will the scientific communities accept preprints without peer review when this process itself has been entangled with a lack of incentives, credits, and recognition for peer reviewers [36,75,76]. Since preprints are not necessarily peer reviewed and explicit about that, this remains to be discussed.…”

“…ASAPbio's initiative is worth mentioning for accelerating scholarly communication in life sciences through preprints. There is also an equal emphasis on standards, research integrity and ethics, quality, and credibility to navigate through the peer review process with scope for new initiatives having potential issues and advantages disrupting scholarly communication both in systems and as a process with incentives in place of fostering open research environments and open access publishing [76,79].…”

Selvitys avoimesta vertaisarvioinnista kotimaisen tiedejulkaisemisen kentällä

“…Historiallisesta perspektiivistä katsottuna vertaisarviointi on ollut keskeinen osa tieteellisen tiedon tuottamista ja julkaisemista jo useamman vuosisadan ajan. Prosessi toimi alkuun tieteellisten seurojen sisällä, mutta on sittemmin tieteen tekemisen laajentumisen ja julkaisemisen kansainvälistymisen myötä levinnyt myös niiden ulkopuolelle (Tennant et al 2017).…”

“…Huomioon on kuitenkin otettava, että jo avoimen vertaisarvioinnin käsite itsessään on monitahoinen ja vaatii tarkennusta. Keskeistä on, että avointa vertaisarviointia ei voi lähestyä yhtenä staattisena, historiattomana kokonaisuutena (Tennant et al 2017). Vertaisarviointi on käsite, ei metodi, ja sellaisenaan monikäyttöinen (Görögh et al 1 Avoimen tieteen teemoihin lasketaan avoimen vertaisarvioinnin ohella avoin data, open access (tutkimustulosten avoin saatavuus), avoin lähdekoodi, avoin metodologia sekä avoimet koulutusresurssit.…”

Who reviews for predatory journals? A study on reviewer characteristics

“…A recent study suggests that 13.7 million reviews are done every year and that writing a review takes a median five hours [1]. When assessing a manuscript for its soundness, originality, validity, and possible impact, reviewers serve a critical gatekeeping function: they help ensure that only papers, which pass a certain quality threshold, enter the scholarly record [2,3]. Such gatekeeping is particularly important in fields where practitioners and policymakers rely on evidence in the form of published journal articles.…”