Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial

Abstract

Resumen. Los procesos de evaluación de las revistas académicas y científicas se han transformado en los últimos años debido al incremento tanto del número de revistas como del volumen de artículos recibidos. Además, el uso de internet ha generado un fuerte aperturismo de la ciencia, con cada vez más revistas en acceso abierto. Los equipos editoriales han tenido que adaptarse y decidirse por un modelo de evaluación que se adecue a este nuevo entorno. Dado este contexto, en este artículo se expone el caso particular del sistema de evaluación de Encrucijadas. Revista Crítica de Ciencias Sociales. Esta revista plantea un modelo de evaluación abierta en el que se conozcan tanto autores como evaluadores durante todo el proceso de revisión. Mediante un cuestionario precodificado semiabierto, se analiza cuál fue la percepción sobre ese sistema de evaluación tanto de autores como evaluadores. La conclusión principal es que hay una aceptación del modelo por parte de los dos colectivos, si bien hay ligeras diferencias por edad, sexo y colectivo Palabras clave: revistas, proceso editorial, evaluación por pares abierta, evaluación por doble ciego.[en] Potentiality and feasibility of open peer review: the case of Encrucijadas. Revista Crítica de Ciencias Sociales Abstract. Evaluation processes of scientific and academic journals have been transformed in recent years due to the increase in both the number of journals and the volume of papers received. Furthermore, the use of Internet has generated a strong openness of science, with more and more open access journals. Editorial teams have had to adapt to this new setting and decide on an evaluation model. In this context, this paper presents the case of the evaluation system of the Spanish journal Encrucijadas. Revista Crítica de Ciencias Sociales. This journal proposes an open peer review in which both authors and reviewers are known each other during all the evaluation process. We use a precoded semi-_____________ 1 Socióloga, profesional independiente

What is open peer review? A systematic review

“…Reviewer anonymity, meanwhile, is presumed to protect reviewers from undue influence, allowing them to give candid feedback without fear of possible reprisals from aggrieved authors. Various studies have failed to show that such measures increase review quality, however ( Fisher et al , 1994; Godlee et al , 1998; Justice et al , 1998; McNutt et al , 1990; van Rooyen et al , 1999). As Godlee and her colleagues have said, “Neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on rate of detection of errors.…”

“…Proponents of open identity peer review argue that it will enhance accountability, further enable credit for peer reviewers, and simply make the system fairer: “most importantly, it seems unjust that authors should be “judged” by reviewers hiding behind anonymity” ( van Rooyen et al , 1999). Open identity peer review is argued, moreover, to potentially increase review quality, as it is theorised that reviewers will be more highly motivated and invest more care in their reviews if their names are attached to them.…”

“…Opponents counter this by arguing that signing will lead to poorer reviews, as reviewers temper their true opinions to avoid causing offence. To date, studies have failed to show any great effect in either direction ( McNutt et al , 1990; van Rooyen et al , 1999; van Rooyen et al , 2010). However, since these studies derive from only one disciplinary area (medicine), the results cannot taken as representative and hence further research is undoubtedly required.…”

Peer review in medical journals: Beyond quality of reports towards transparency and public scrutiny of the process

“…According to its detractors, open review may thus result in worse reports compared 137 to blind review, but this has not been observed in randomized, controlled trials [10,11,27]. 138…”

A multi-disciplinary perspective on emergent and future innovations in peer review

“…This trial was repeated on a larger scale by Justice et al (1998) and Van Rooyen et al (1999), with neither study finding that blinding reviewers improved the quality of reviews. These studies also showed that blinding is difficult in practice, as many manuscripts include clues on authorship.…”