Restrictions and their reporting in systematic reviews of effectiveness: an observational study

Abstract

Background Restrictions in systematic reviews (SRs) can lead to bias and may affect conclusions. Therefore, it is important to report whether and which restrictions were used. This study aims to examine the use of restrictions regarding language, publication period, and study type, as well as the transparency of reporting in SRs of effectiveness. Methods A retrospective observational study was conducted with a random sample of 535 SRs of effectiveness indexed in PubMed between 2000 and 2019. The use of restrictions and their reporting were analysed using descriptive statistics. Results Of the total 535 SRs included, four out of every ten (41.3%) lacked information on at least one of the three restrictions considered (language, publication period, or study type). Overall, 14.6% of SRs did not provide information on restrictions regarding publication period, 19.1% regarding study type, and 18.3% regarding language. Of all included SRs, language was restricted in 46.4%, and in more than half of the SRs with restricted language (130/248), it was unclear whether the restriction was applied during either the search or the screening process, or both. The restrictions were justified for publication period in 22.2% of the respective SRs (33/149), study type in 6.5% (28/433), and language in 3.2% (8/248). Differences in reporting were found between countries as well as between Cochrane and non-Cochrane reviews. Conclusions This study suggests that there is a lack of transparency in reporting on restrictions in SRs. Authors as well as editors and reviewers should be encouraged to improve the reporting and justification of restrictions to increase the transparency of SRs.

Investigating the use of actor-network theory in healthcare: a protocol for a systematic review

“…To capture recent contextual developments, trends and advancements in the theory, a 10-year publication restriction will be used as part of the search criteria. 42 …”

Mental Health on Survivor of Natural Disaster: Literature Review

“…The search used Google Scholar, PubMed, and Scopus databases. Researchers only chose journals published in English and 2018-2023 in order to include only recent evidence and update the review (Helbach et al, 2022). Literature searches were carried out using the Publish and Perish application.…”

Professional Development in Reading Comprehension: A Meta‐analysis of the Effects on Teachers and Students

“…Additionally, no restriction was used for publication year. Restrictions on publication year should be justified (e.g., updating a previous review, reflecting the start of scientific relevance; Helbach et al., 2022). Given that no previous reviews have specifically examined reading comprehension PD effects and no date of scientific relevance could be identified (e.g., a significant change to how reading comprehension is taught), publications across all years were included in the search.…”