Heterogeneity of studies in anesthesiology systematic reviews: a meta-epidemiological review and proposal for evidence mapping

Abstract

Heterogeneity among the primary studies included in a systematic review (SR) is one of the most challenging considerations for systematic reviewers. Current practices in anaesthesiology SRs have not been evaluated, but traditional methods may not provide sufficient information to evaluate the true nature of these differences. We address these issues by examining the practices for evaluating heterogeneity in anesthesiology reviews. Also, we propose a mapping method for presenting heterogeneous aspects of the primary studies in SRs.We evaluated heterogeneity practices reported in SRs published in highly ranked anesthesiology journals and Cochrane reviews. Elements extracted from the SRs included heterogeneity tests, models used, analyses conducted, plots used, and I2 values. Additionally, we selected a SR to develop an evidence map in order to display clinical heterogeneity.Our statistical analysis showed 150/207 SRs reporting a test for statistical heterogeneity. Plots were used in 138 reviews to display heterogeneity. Subgroup analyses were the most commonly reported analysis (54%). Meta-regression and sensitivity analyses were used sparingly (25%; 23% respectively). A random effects model was most commonly reported (33%). Heterogeneity statistics across meta-analyses suggested that, in our sample, the majority (55%) did not present sufficient heterogeneity to be of great concern. Cochrane reviews (n=58) were also analysed. Plots were used in 88% of Cochrane reviews. Subgroup analysis was used in 59% Cochrane reviews, while sensitivity analysis was used in 62%.Many reviews did not provide sufficient detail regarding heterogeneity. We are calling for improvement to reporting practices.

Efficacy of perioperative dexmedetomidine on postoperative delirium: systematic review and meta-analysis with trial sequential analysis of randomised controlled trials

“…I was used to identify statistical heterogeneity: an I 2 value of 30e60% was considered as moderate heterogeneity; and 50e90% as substantial heterogeneity. 34,35 Odds ratio (OR) and 95% CI were calculated for dichotomous outcomes. Continuous outcomes were calculated by weighted mean differences (WMD) of mean values and standard deviations.…”

Methodological Quality Assessment of Systematic Review or Meta-Analysis Using AMSTAR-2: the Long-Term Effectiveness or Efficacy of Opioids for Chronic Non-Cancer Pain

“…Likewise, in the light of an increase of systematic reviews or meta-analyses including non-randomized studies, it is important to use a different tool to when considering bias 54,55 . Meanwhile, the result of a bias assessment would in uence the statistical heterogeneity of a sample, which would in turn in uence the heterogeneity of SR/metaanalysis and the robustness of result 56 , With this in mind, it was necessary to assess the impact of the ROB of individual studies on the overall con dence of systematic review or meta-analysis.…”

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

“…Many of the authors were from Canadian institutions (n = 12, 38.7%), followed by Japan and USA (n = 4, 12.9% each). Based on the previously defined criteria, we scored ten studies (32.3%) as generalizable [28,30,38,40,49,[51][52][53][54]57]. Only three studies (9.7%, two of which we scored as generalizable) commented on generalizability and reported their own work as generalizable, either to the subject area (e.g., venous ulcer disease), to a clinical area, or in general terms [27,30,38].…”