Characteristics of the sources, evaluation, and grading of the certainty of evidence in systematic reviews in public health: A methodological study

Abstract

ObjectivesTo systematically explore how the sources of evidence, types of primary studies, and tools used to assess the quality of the primary studies vary across systematic reviews (SRs) in public health.MethodsWe conducted a methodological survey of SRs in public health by searching the of literature in selected journals from electronic bibliographic databases. We selected a 10% random sample of the SRs that met the explicit inclusion criteria. Two researchers independently extracted data for analysis.ResultsWe selected 301 SRs for analysis: 94 (31.2%) of these were pre-registered, and 211 (70.1%) declared to have followed published reporting standard. All SRs searched for evidence in electronic bibliographic databases, and more than half (n = 180, 60.0%) searched also the references of the included studies. The common types of primary studies included in the SRs were primarily cross-sectional studies (n = 132, 43.8%), cohort studies (n = 126, 41.9%), randomized controlled trials (RCTs, n = 89, 29.6%), quasi-experimental studies (n = 83, 27.6%), case-control studies (n = 58, 19.3%) qualitative studies (n = 38, 12.6%) and mixed-methods studies (n = 32, 10.6%). The most frequently used quality assessment tools were the Newcastle-Ottawa Scale (used for 50.0% of cohort studies and 55.6% of case-control studies), Cochrane Collaboration's Risk of Bias tool (50.7% of RCTs) and Critical Appraisal Skills Program (38.5% of qualitative studies). Only 20 (6.6%) of the SRs assessed the certainty of the body of evidence, of which 19 (95.0%) used the GRADE approach. More than 65% of the evidence in the SRs using GRADE was of low or very low certainty.ConclusionsSRs should always assess the quality both at the individual study level and the body of evidence for outcomes, which will benefit patients, health care practitioners, and policymakers.

Epidemiology and Reporting Characteristics of Systematic Reviews in Orthopedic Journals: A Meta-Epidemiological Study

“…In 2022, only 12% of the SRs used PRISMA 2020, indicating that the authors could not comply with the updated reporting guidelines. Only approximately 10% of SRs consider the certainty of evidence, similar to other fields [20,21]. Interpreting the results based on the certainty of evidence rather than focusing only on the statistical significance of the effect estimates is essential.…”

The Assessment of SF-36 Survey for Quality-of-Life Measurement after Radical Cystectomy for Muscle-Invasive Bladder Cancer: A Systematic Review

“…To assess the quality of the studies included in our review, we utilized the Newcastle–Ottawa Scale for evaluating cohort studies and the Cochrane Collaboration’s tool for assessing randomized trials [ 20 ]. Two researchers independently conducted the evaluations, assigning scores that defined the studies’ quality as either low, medium, or high.…”

Attention-deficit/hyperactivity disorder as a risk factor for being involved in intimate partner violence and sexual violence: a systematic review and meta-analysis

“…This scale evaluates observational studies on three different domains, namely: (1) selection, (2) comparability, and (3) exposure. It has a maximum score of nine stars, and any study with a score of six or less was considered to be at a high risk of bias (Xun et al, 2023). The NOS has been modified in the literature to accommodate case–control, cohort and cross-sectional studies (Xun et al, 2023).…”

“…It has a maximum score of nine stars, and any study with a score of six or less was considered to be at a high risk of bias (Xun et al, 2023). The NOS has been modified in the literature to accommodate case–control, cohort and cross-sectional studies (Xun et al, 2023).…”