Librarians and information specialists as methodological peer-reviewers: a case-study of the International Journal of Health Governance

Abstract

Background Objectives of this study were to analyze the impact of including librarians and information specialist as methodological peer-reviewers. We sought to determine if and how librarians’ comments differed from subject peer-reviewers’; whether there were differences in the implementation of their recommendations; how this impacted editorial decision-making; and the perceived utility of librarian peer-review by librarians and authors. Methods We used a mixed method approach, conducting a qualitative analysis of reviewer reports, author replies and editors’ decisions of submissions to the International Journal of Health Governance. Our content analysis categorized 16 thematic areas, so that methodological and subject peer-reviewers’ comments, decisions and rejection rates could be compared. Categories were based on the standard areas covered in peer-review (e.g., title, originality, etc.) as well as additional in-depth categories relating to the methodology (e.g., search strategy, reporting guidelines, etc.). We developed and used criteria to judge reviewers’ perspectives and code their comments. We conducted two online multiple-choice surveys which were qualitatively analyzed: one of methodological peer-reviewers’ perceptions of peer-reviewing, the other of published authors’ views on the suggested revisions. Results Methodological peer-reviewers assessed 13 literature reviews submitted between September 2020 and March 2023. 55 reviewer reports were collected: 25 from methodological peer-reviewers, 30 from subject peer-reviewers (mean: 4.2 reviews per manuscript). Methodological peer-reviewers made more comments on methodologies, with authors more likely to implement their changes (52 of 65 changes, vs. 51 of 82 by subject peer-reviewers); they were also more likely to reject submissions (seven vs. four times, respectively). Where there were differences in recommendations to editors, journal editors were more likely to follow methodological peer-reviewers (nine vs. three times, respectively). The survey of published authors (87.5% response rate) revealed four of seven found comments on methodologies helpful. Librarians’ survey responses (66.5% response rate) revealed those who conducted peer-reviews felt they improved quality of publications. Conclusions Librarians can enhance evidence synthesis publications by ensuring methodologies have been conducted and reported appropriately. Their recommendations helped authors revise submissions and facilitated editorial decision-making. Further research could determine if sharing reviews with subject peer-reviewers and journal editors could benefit them in better understanding of evidence synthesis methodologies.

Enhancing the quality and efficiency of regulatory science literature reviews through innovation and collaboration with library and information science experts

“…However, with a multitude of databases and variations in search terms, understanding how and where to identify relevant data is not a simple task and requires strategy and knowledge of data systems. Therefore, studies that choose to include those with library and information science expertise on their teams are at a significant advantage for producing high quality literature reviews with less overall effort ( 27 – 29 ).…”

“…Indeed, a whole field of library and information science has evolved to develop methods for the management and discoverability of research data for research synthesis, whether in the published literature or in data repositories ( 30 ). Those skilled in information science can provide invaluable insights into the best approaches to identify potentially relevant literature ( 27 – 29 ). Information science experts can improve how controlled vocabulary can be used or improved for a specific research topic, as well as generate high quality search strategies.…”