On ratio measures of heterogeneity for meta‐analyses

Abstract

As a measure of heterogeneity in meta‐analysis, the coefficient of variation (CV) has been recently considered, providing researchers with a complement to the very popular I2 measure. While I2 measures the proportion of total variance that is due to variance of the random effects, the CV is the ratio of the standard deviation of the random effects to the effect of interest. Consequently, the CV provides a different measure of the extent of heterogeneity in a meta‐analysis relative to the effect being measured. However, very large CV values can occur when the effect is small making interpretation difficult. The purpose of this article is two‐fold. First, we consider variants of the CV that exist in the interval (]0,0.5em1 which may be preferable for some researchers. Second, we provide interval estimators for the CV and its variants with excellent coverage properties. We perform simulation studies based on simulated and real data sets and draw comparisons between the methods. For both the CV and its transformations, we recommend confidence intervals using the propagating imprecision method or, as a simpler alternative but at the expense of slightly worse performance in terms of coverage, combining reduced‐coverage confidence intervals for the two parameters. These interval estimators typically have better coverage properties for the CV measure than those previously considered.

Extending the coefficient of variation for measuring heterogeneity following a meta-regression

“…We have used the CV for fixed values of the moderator, as well as means of the CV over several moderator values. We also considered transformed values of the CV and associated CIs as was considered by Cairns & Prendergast (2020). It is of interest to be able to assess how much adding moderators affects the amount of explained heterogeneity of a meta-analysis relative to the size of the effect.…”

“…Additionally, given the prominent use of I 2 , that is on a simple-to-understand scale, not everyone may be comfortable reporting CV B . Therefore, Cairns & Prendergast (2020) transformed the CV as…”

“…Takkouche et al (2013) considered Wald CIs for the log of CV B . For the estimator of M 1 (and M 2 ), Cairns & Prendergast (2020) showed that improved intervals were obtained when these were transformed using the logit transformation and some different interval methods to be discussed later. Hence, using their variance of the logit transformed estimation and also the link between this and the log of the CV estimator shown in (6), we have…”

“…, remembering that β is referring to β x = β 0 + β 1 x. The full PropImp algorithm for the CV is discussed in Cairns & Prendergast (2020).…”

Some Contributions to Measuring and Understanding Heterogeneity in Meta-Analysis

“…We then propose new confidences for the CV and its transformations which exhibit strong coverage properties, providing improvements over existing intervals for the CV. We show that the use of such measures, together with confidence intervals alongside , provides a more comprehensive look at the amount of heterogeneity in the meta-analysis [3]. We then show that these concepts can extend to the context of meta-regression retaining their strong coverage properties [2].…”