Peer Review of Reviewers: The Author’s Perspective

Abstract

The aim of this study was to investigate the opinion of authors on the overall quality and effectiveness of reviewers’ contributions to reviewed papers. We employed an on-line survey of thirteen journals which publish articles in the field of life, social or technological sciences. Responses received from 193 authors were analysed using a mixed-effects model in order to determine factors deemed the most important in the authors’ evaluation of the reviewers. Qualitative content analysis of the responses to open questions was performed as well. The mixed-effects model revealed that the authors’ assessment of the competence of referees strongly depended on the final editorial decision and that the speed of the review process was influential as well. In Ordinary Least Squares (OLS) analysis on seven questions detailing authors’ opinions, perception of review speed remained a significant predictor of the assessment. In addition, both the perceived competence and helpfulness of the reviewers significantly and positively affected the authors’ evaluation. New models were used to re-check the value of these two factors and it was confirmed that the assessment of the competence of reviewers strongly depended on the final editorial decision.

Is the quality of reviews reflected in editors' and authors' satisfaction with peer review? A cross‐sectional study in 12 journals across four research fields

“…Our study also showed that the review reports in diverse disciplines that recommended manuscript acceptance rather than rejection were associated with higher author satisfaction scores regarding the constructiveness of the received reviews, assuming that the authors were unaware of the actual decision. Previous studies have shown that authors preferred acceptance recommendations when they were cognisant of the final decision on their manuscript submitted to a single medical journal (Gibson et al ., 2008; Weber et al ., 2002) or journals in various fields (Drvenica, Bravo, Vejmelka, Dekanski, & Nedić, 2019). Even though reviewers' recommendations (accepted, rejected, revise) were not available to the investigators in our study, it is likely that they perceived them based on the content, language, or tone of the reviews.…”

Reading Peer Review

“…Aiming, with substantial European research funding, to 'improve [the] efficiency, transparency and accountability of peer review through a trans-disciplinary, cross-sectorial collaboration', the consortium has been one of the most prolific centres for research into peer review in the past half decade. Publications from the group have spanned the author perspective on peer review (Drvenica et al 2019), the reward systems of peer review (Zaharie and Seeber 2018), the links between reputation and peer review (Grimaldo, Paolucci, and Sabater-Mir 2018), the role that artificial intelligence might play in future structures of review (Mrowinski et al 2017), the timescales involved in review (Huisman and Smits 2017;Mrowinski et al 2016), the reasons why people cite retracted papers (Bar-Ilan and Halevi 2017), the fate of rejected manuscripts (Casnici, Grimaldo, Gilbert, Dondio et al 2017), and the ways in which referees act in multidisciplinary contexts .…”

“…For example, when asking for specific reruns of experiments, terms such as 'perhaps' or 'probably' indicated a level of uncertainty in the recommendation that could lead to such a classification. While there is no universally agreed upon definition of 'hedging' (Crompton 1997;Hyland 1996Hyland , 1998Hyland , 2000Hyland , 2004Myers 1989;Nash 1990;Prasithrathsint 2015;Salager-Meyer 1994), previously it has been considered as a strategy for politeness (for instance, Brown and Levinson 1987;Dressen-Hammouda 2013;Fernández 2005;Held 2010; Spencer-Oatey and Franklin 2009; Tang 2013; Varttala 2001) with cross-cultural and gender differences in usage (Coates 2013;Hinkel 2005;Scollon and Scollon 2001). In academic writing, hedges imply 'that a statement is based on plausible reasoning rather than certain knowledge' (Hyland 1998, 4), allowing the reader either to become complicit in accepting such a statement or to contest it (Kim and Lim 2015).…”

A framework for assessing the peer review duration of journals: case study in computer science

“…Yet, mind that the data gathered manually from authors are certainly different from the ones extracted automatically. The former are subjective, thus not 100% reliable (author's impressions of the review process are strongly dependent on the final editorial decision and the speed of the review process [10]), but obviously more extensive, whereas the latter are more reliable but less extensive (typically only dates).…”