Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review

Abstract

Background The emergence of systems based on large language models (LLMs) such as OpenAI’s ChatGPT has created a range of discussions in scholarly circles. Since LLMs generate grammatically correct and mostly relevant (yet sometimes outright wrong, irrelevant or biased) outputs in response to provided prompts, using them in various writing tasks including writing peer review reports could result in improved productivity. Given the significance of peer reviews in the existing scholarly publication landscape, exploring challenges and opportunities of using LLMs in peer review seems urgent. After the generation of the first scholarly outputs with LLMs, we anticipate that peer review reports too would be generated with the help of these systems. However, there are currently no guidelines on how these systems should be used in review tasks. Methods To investigate the potential impact of using LLMs on the peer review process, we used five core themes within discussions about peer review suggested by Tennant and Ross-Hellauer. These include 1) reviewers’ role, 2) editors’ role, 3) functions and quality of peer reviews, 4) reproducibility, and 5) the social and epistemic functions of peer reviews. We provide a small-scale exploration of ChatGPT’s performance regarding identified issues. Results LLMs have the potential to substantially alter the role of both peer reviewers and editors. Through supporting both actors in efficiently writing constructive reports or decision letters, LLMs can facilitate higher quality review and address issues of review shortage. However, the fundamental opacity of LLMs’ training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports. Additionally, as editorial work has a prominent function in defining and shaping epistemic communities, as well as negotiating normative frameworks within such communities, partly outsourcing this work to LLMs might have unforeseen consequences for social and epistemic relations within academia. Regarding performance, we identified major enhancements in a short period and expect LLMs to continue developing. Conclusions We believe that LLMs are likely to have a profound impact on academia and scholarly communication. While potentially beneficial to the scholarly communication system, many uncertainties remain and their use is not without risks. In particular, concerns about the amplification of existing biases and inequalities in access to appropriate infrastructure warrant further attention. For the moment, we recommend that if LLMs are used to write scholarly reviews and decision letters, reviewers and editors should disclose their use and accept full responsibility for data security and confidentiality, and their reports’ accuracy, tone, reasoning and originality.

Can Artificial Intelligence Be My Coauthor?

“…The conversational prowess of AI authoring tools could facilitate the editorial and peer reviewer process by helping to craft summaries and even decisions more quickly 23 . Some have even suggested that these tools could help address a shortage of peer reviewers 24 . However, Hosseini and Horbach 24 expressed concerns about the opacity of proprietary models and their training data, inner workings, and the potential for exacerbating existing biases.…”

“…Some have even suggested that these tools could help address a shortage of peer reviewers 24 . However, Hosseini and Horbach 24 expressed concerns about the opacity of proprietary models and their training data, inner workings, and the potential for exacerbating existing biases. Thus, they argue that like authors, reviewers, and editors should also disclose their use.…”

Guidance for Authors, Peer Reviewers, and Editors on Use of AI, Language Models, and Chatbots

“…However, they concluded that "the fundamental opacity of LLMs' training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports." 4 JAMA and the JAMA Network journals have extended the policy on use of AI tools to peer reviewers with recommendations for responsible and accountable use and reminders of the confidential nature of submitted manuscripts and the peer review process. These journals use a single-anonymized review process, in which peer reviewer identities are kept confidential (unless reviewers choose to reveal their names in their formal reviews), author identities are made known to reviewers, and submitted manuscripts and other content are kept confidential if and until publication.…”

“…Hosseini and Horbach recently reviewed the potential use of LLMs in the peer review process following Tennant and Ross-Hellauer’s 5 core themes about peer review: the reviewers’ role, the editors’ role, functions and quality of peer reviews, reproducibility, and the social and epistemic functions of peer reviews. They identified potential benefits of these tools to improve efficiency and productivity in the editorial process and peer review and help with reviewer fatigue.…”

“…They identified potential benefits of these tools to improve efficiency and productivity in the editorial process and peer review and help with reviewer fatigue. However, they concluded that “the fundamental opacity of LLMs’ training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports.”…”