Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers

Abstract

Open peer review (OPR) is a cornerstone of the emergent Open Science agenda. Yet to date no large-scale survey of attitudes towards OPR amongst academic editors, authors, reviewers and publishers has been undertaken. This paper presents the findings of an online survey, conducted for the OpenAIRE2020 project during September and October 2016, that sought to bridge this information gap in order to aid the development of appropriate OPR approaches by providing evidence about attitudes towards and levels of experience with OPR. The results of this cross-disciplinary survey, which received 3,062 full responses, show the majority (60.3%) of respondents to be believe that OPR as a general concept should be mainstream scholarly practice (although attitudes to individual traits varied, and open identities peer review was not generally favoured). Respondents were also in favour of other areas of Open Science, like Open Access (88.2%) and Open Data (80.3%). Among respondents we observed high levels of experience with OPR, with three out of four (76.2%) reporting having taken part in an OPR process as author, reviewer or editor. There were also high levels of support for most of the traits of OPR, particularly open interaction, open reports and final-version commenting. Respondents were against opening reviewer identities to authors, however, with more than half believing it would make peer review worse. Overall satisfaction with the peer review system used by scholarly journals seems to strongly vary across disciplines. Taken together, these findings are very encouraging for OPR’s prospects for moving mainstream but indicate that due care must be taken to avoid a “one-size fits all” solution and to tailor such systems to differing (especially disciplinary) contexts. OPR is an evolving phenomenon and hence future studies are to be encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

“…1b respondents would not consider it: 18 gave scores of 1 or 2 (mean = 2.23, median = 2). This mirrors the qualitative feedback regarding the possibility of such a process incurring retaliation for the reviewers of a rejected paper, for example, and echoes previous work (e.g., [9,54]). Several possible procedures for non-anonymous reviews exist beyond simply asking reviewers to sign their reviews, however, such as giving the names of reviewers without attaching them to any specific report or only publishing the names of reviewers of accepted papers.…”

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

“…However, concerning the possibility to sign reviews it seems that most respondents would not consider it: 18 gave scores of 1 or 2 (mean = 2.23, median = 2). This mirrors the qualitative feedback regarding the possibility of such a process incurring retaliation for the reviewers of a rejected paper, for example, and echoes previous work (e.g., [8,51]). Several possible procedures for non-anonymous reviews exist beyond simply asking reviewers to sign their reviews, however, such as giving the names of reviewers without attaching them to any specific report or only publishing the names of reviewers of accepted papers.…”

Purpose-oriented approach to the reader’s assessment of the quality of research papers

“…Другая проблемная зона в сфере редакционно-издательской деятельности -институт рецензирования научного контента. Конечно, издатели и редакторы научных журналов непрерывно работают над совершенствованием процедуры научной экспертизы поступающих в редакцию рукописей, но существующие практики подвергаются критике в связи с их часто субъективным, предвзятым или недобросовестным характером [Kulkarni, 2016;Tennant et al, 2017;Ross-Hellauer et al, 2017]. Хотя анонимное рецензирование во многом остается основным подходом [Mulligan et al,…”