Identifying the characteristics of excellent peer reviewers by using Publons

Abstract

PurposeThis study aimed to identify the characteristics of excellent peer reviewers by using Publons.com (an open and free online peer review website).Design/methodology/approachReviewers of the clinical medicine field on Publons were selected as the sample (n = 1,864). A logistic regression model was employed to examine the data.FindingsThe results revealed that reviewers' verified reviews, verified editor records, and whether they were the Publons mentors had significant and positive associations with excellent peer reviewers, while their research performance (including the number of articles indexed by Web of Science (WOS), citations, H-index and high-cited researcher), genders, words per review, number of current/past editorial boards, whether they had experiences of post-publication review on Publons and whether they were Publons academy graduates had no significant associations with excellent peer reviewers.Originality/valueThis study could help journals find excellent peer reviewers from free and open online platforms.Peer reviewThe peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-11-2021-0604.

The Role of Publons in the Context of Open Peer Review

“…Finally, some authors have sought to define the parameters that would allow an “excellent” Publons-associated peer reviewer to be distinguished, focusing on their productivity and bibliometric metrics and performance associated with their own citations, number of articles in WoS, number of words in a peer review report, and some other factors [ 57 ]. Similarly, Zhang et al [ 59 ] searched for associations between review length and several factors linked to the peer reviewers themselves (gender, cultural background, disciplines, English proficiency, publications, and verified reviews).…”

“…Not many of these papers focused on the skill-sets of peer reviewers as being a key factor in the success of the peer review process, and how such skills might have factored into Publons-based recognition, select aspects of which were more recently assessed by Huang and Zong [ 57 ], Lei [ 58 ] and Zhang et al [ 59 ]. For example, many librarians and information professionals have a very high set of organizational and analytical skills that would make them ideal peer reviewers of systematic reviews, with a survey of 291 such academics indicating that the vast majority (95%) recommending rejection or revisions of papers based on methodological flaws [ 60 ].…”

“…Under-appreciation of peer reviewers' voluntary work is certainly one logical reason why reviewers might be motivated not to peer review, but there are also costs, namely "having to deal with low-quality papers, taking responsibility for errors, and editors and authors disregarding the review suggestions" (p. 220) [52]. Not many of these papers focused on the skill-sets of peer reviewers as being a key factor in the success of the peer review process, and how such skills might have factored into Publons-based recognition, select aspects of which were more recently assessed by Huang and Zong [57], Lei [58] and Zhang et al [59]. For example, many librarians and information professionals have a very high set of organizational and analytical skills that would make them ideal peer reviewers of systematic reviews, with a survey of 291 such academics indicating that the vast majority (95%) recommending rejection or revisions of papers based on methodological flaws [60].…”