The relationship of polarity of post-publication peer review to citation count

Abstract

PurposeThe purpose of this study is to investigate the relationship of the post-publication peer review (PPPR) polarity of a paper to that paper's citation count.Design/methodology/approachPapers with PPPRs from Publons.com as the experimental groups were manually matched 1:2 with the related papers without PPPR as the control group, by the same journal, the same issue (volume), the same access status (gold open access or not) and the same document type. None of the papers in the experimental group or control group received any comments or recommendations from ResearchGate, PubPeer or F1000. The polarity of the PPPRs was coded by using content analysis. A negative binomial regression analysis was conducted to examine the data by controlling the characteristics of papers.FindingsThe four experimental groups and their corresponding control groups were generated as follows: papers with neutral PPPRs, papers with both negative and positive PPPRs, papers with negative PPPRs and papers with positive PPPRs as well as four corresponding control groups (papers without PPPRs). The results are as follows: while holding the other variables (such as page count, number of authors, etc.) constant in the model, papers that received neutral PPPRs, those that received negative PPPRs and those that received both negative and positive PPPRs had no significant differences in citation count when compared to their corresponding control pairs (papers without PPPRs). Papers that received positive PPPRs had significantly greater citation count than their corresponding control pairs (papers without PPPRs) while holding the other variables (such as page count, number of authors, etc.) constant in the model.Originality/valueBased on a broader range of PPPR sentiments, by controlling many of the confounding factors (including the characteristics of the papers and the effects of the other PPPR platforms), this study analyzed the relationship of various polarities of PPPRs to citation count.

The Role of Publons in the Context of Open Peer Review

“…Finally, and also importantly, given that science, and its replication, may be in a state of crisis, including the core principle of industry-wide trust, it would be important to evaluate the post-publication peer reviewer (PPPR) reports that existed at Publons [ 93 ], and to appreciate their content in WoS to assess whether positive PPPR reports (praising research) outweigh negative ones (criticizing research or calling out errors or possible fraud). Goldstein [ 94 ] noted a deficit of PPPR reports relative to pre-publication reports at Publons, while also noting bias towards, or in favor of, life sciences (health and medical sciences), while Publons metrics were weakly and insignificantly correlated with bibliometric and altmetric indicators.…”

Identifying the characteristics of excellent peer reviewers by using Publons

“…This might be because some post-publication peer reviews on Publons.com were not real post-publication peer reviews. A previous study revealed that some post-publication peer reviews on Publons.com were structured like pre-publication peer reviews (Zong et al. , 2020).…”

“…This might be because some post-publication peer reviews on Publons.com were not real post-publication peer reviews. A previous study revealed that some post-publication peer reviews on Publons.com were structured like prepublication peer reviews (Zong et al, 2020). That is, reviewers might transfer their prepublication peer reviews of the articles they reviewed to the post-publication peer review of the articles on Publons.com.…”

Which structure of academic articles do referees pay more attention to?: perspective of peer review and full-text of academic articles

“…The jointed width and depth component model was proposed, and the PRC was represented with a matching mechanism to predict the citations. Zong et al. (2020a, b) explored whether the sentiment polarity of post-publication PRC from PubPeer, F1000 and ResearchGate affected the citations through content analysis method, control variable method, negative binomial regression analysis and other methods.…”

“…In recent years, open peer review has attracted more and more scholars' attention. Some scholars found that the citations of papers in open access journals are higher than that in non-open access journals (Vanclay, 2013), and open peer review can improve the citations (Zong et al. , 2020a, b).…”