Quantifying heterogeneity in a meta‐analysis

Abstract

The extent of heterogeneity in a meta-analysis partly determines the difficulty in drawing overall conclusions. This extent may be measured by estimating a between-study variance, but interpretation is then specific to a particular treatment effect metric. A test for the existence of heterogeneity exists, but depends on the number of studies in the meta-analysis. We develop measures of the impact of heterogeneity on a meta-analysis, from mathematical criteria, that are independent of the number of studies and the treatment effect metric. We derive and propose three suitable statistics: H is the square root of the chi2 heterogeneity statistic divided by its degrees of freedom; R is the ratio of the standard error of the underlying mean from a random effects meta-analysis to the standard error of a fixed effect meta-analytic estimate, and I2 is a transformation of (H) that describes the proportion of total variation in study estimates that is due to heterogeneity. We discuss interpretation, interval estimates and other properties of these measures and examine them in five example data sets showing different amounts of heterogeneity. We conclude that H and I2, which can usually be calculated for published meta-analyses, are particularly useful summaries of the impact of heterogeneity. One or both should be presented in published meta-analyses in preference to the test for heterogeneity.

Sex‐Specific Associations Between Alcohol Consumption and Incidence of Hypertension: A Systematic Review and Meta‐Analysis of Cohort Studies

“…Small‐study bias was examined using Egger's regression‐based test 37. Variation in the effect size because of heterogeneity between‐studies was quantified using the I 2 statistic 38. Between‐study heterogeneity was investigated with random‐effects metaregressions 39…”

Intravenous Thrombolysis for Acute Ischemic Stroke in Patients Receiving Antiplatelet Therapy: A Systematic Review and Meta‐analysis of 19 Studies

“…ORs and both the lower and upper limits of corresponding 95% CIs from each study were logarithmically transformed to normalize distribution before they were combined. Heterogeneity was assessed by using the χ 2 test and quantified by using the I 2 statistic, which represents the percentage of total variation across studies that is attributable to heterogeneity rather than chance 33. Subgroup analyses were carried out based on outcome definitions, onset‐to‐treatment time, specific antiplatelet agents, and the NOS scores of included studies.…”

Blood concentrations of carotenoids and retinol and lung cancer risk: an update of the WCRF–AICR systematic review of published prospective studies

“…Random effect models were used to calculate the summary RRs and 95% CIs to take into account heterogeneity across studies 18. Heterogeneity was determined using Q and I ² statistics 19, and was explored in stratified analyses when there were eight or more studies in the analysis.…”

“…A potential nonlinear dose–response association between blood concentrations of carotenoids and retinol was assessed using fractional polynomial model 19 and the best‐fitting second‐order fractional polynomial regression model, defined as the one with the lowest deviance was determined. A two‐tailed P  < 0.05 was considered statistically significant.…”