How should we handle predatory journals in evidence synthesis? A descriptive survey‐based cross‐sectional study of evidence synthesis experts

Abstract

Synthesizers of evidence are increasingly likely to encounter studies published in predatory journals during the evidence synthesis process. The evidence synthesis discipline is uniquely positioned to encounter novel concerns associated with predatory journals. The objective of this research was to explore the attitudes, opinions, and experiences of experts in the synthesis of evidence regarding predatory journals. Employing a descriptive survey-based cross-sectional study design, these experts were asked a series of questions regarding predatory journals to explore these attitudes, opinions, and experiences.Two hundred and sixty four evidence synthesis experts responded to this survey. Most respondents agreed with the definition of a predatory journal (86%), however several (19%) responded that this definition was difficult to apply practically. Many respondents believed that studies published in predatory journals are still eligible for inclusion into an evidence synthesis project. However, this was only after the study had been determined to be 'high-quality' (39%) or if the results were validated (13%).

Editorial actions taken to reduce publishing references from predatory sources: A case study

“…Concerns about articles from predatory publishers being included in literature reviews, particularly meta‐analyses, have been raised in the literature (Barker et al, 2023; Lalu et al, 2022; Rice et al, 2021). We found that two of three meta‐analyses in our sample did have these inappropriate references.…”

“…We considered all points of view when deciding to revise our policy. Including studies published by predatory publishers poses a grave threat to the reliability of meta‐analyses with potential real‐world severe consequences, as meta‐analyses are often used to develop clinical protocols and standards (Barker et al, 2023; Lalu et al, 2022; Rice et al, 2021). ‘Developing appropriate methods to account for eligible research published in predatory journals is needed to decrease the potential negative impact of predatory journals on healthcare’ (Rice et al, 2021, p. 3).…”

Reviews and Reviewing: Approaches to Research Synthesis. An Annual Review of Information Science and Technology (ARIST) paper

“…Hayden (2020) cautions that inclusion of articles in systematic reviews from journals lacking rigorous peer review could distort the evidence base. Barker et al (2023) carried out a survey to explore attitudes and experiences of experts in evidence synthesis regarding predatory journals. Survey results demonstrated a lack of consensus regarding the appropriate methods to follow when considering including studies from a predatory journal as part of an evidence synthesis project.…”

Development of the Individual Participant Data (IPD) Integrity Tool for assessing the integrity of randomised trials using individual participant data

“…39 To achieve this, they should routinely screen manuscripts for integrity issues, 40 and we support arguments to require sharing of raw data prior to publication. 10,31 Among journal readers and reviewers, a healthy scepticism is also encouraged; while publications were commonly accepted as truth in the past, as Moore et al argue "the biggest mistake of all, then, is taking evidence on trust and without checking it." 41 In future, we aim to explore how this tool might be adapted and applied to non-randomised and observational studies, as well as emerging novel designs like umbrella and basket trials.…”