Characterizing Peer Review Comments of Academic Articles in Multiple Rounds

Abstract

Peer review can evaluate the quality of academic articles involving the evaluation of some aspects, e.g., methodology, experiment, and aspects are usually the critical sections, substance and properties of the article concerned by reviewers. Previous research on content mining of peer review did not distinguish the round of reviews. Detecting differences peer review comments among different rounds can help us to understand the focus change of the reviewers in the different rounds. This paper takes the Nature Communications as an example to build the corpora of peer reviews in multiple rounds. We correlate review rounds and citations and extract aspects from the corpora to analyze the round characteristics of review comments. We find that there is no significant correlation between the review rounds and the citations frequency, but when the number of rounds is within 1–3, there is a weak positive correlation between the review rounds and citations. Additionally, the reviewers tend to pay more attention to the results analysis and the significance of the work firstly, and then they will focus on the details of the article, such as diagrams. This study can provide new ideas for peer review mining and the application of bibliometrics.