Modelling data from different sites, times or studies: weighted vs. unweighted regression

Fletcher, David; Dixon, Philip M.

doi:10.1111/j.2041-210x.2011.00140.x

Cited by 17 publications

(12 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We suspect that other metrics and similar approaches may suffer from related problems. For example, Fletcher and Dixon (2011) showed that weighted regression (and by extension, probably meta-regression) has lower coverage than unweighted methods, partly due to problems with the estimation of weights. We suggest that future meta-analyses carefully consider whether weights are correlated with effect size estimates and, if so, consider using alternative approaches that avoid this, such as sample size-based approaches (e.g., Hungate et al 2009) or unweighted analyses, although unweighted analyses pose other issues in some cases (e.g., meta-analyses of absolute values [or magnitudes] will be biased if unweighted: Morrissey 2016).…”

Section: Discussionmentioning

confidence: 99%

Bias in meta‐analyses using Hedges’ d

et al. 2018

View full text Add to dashboard Cite

The type of metric and weighting method used in meta‐analysis can create bias and alter coverage of confidence intervals when the estimated effect size and its weight are correlated. Here, we investigate bias associated with the common metric, Hedges’ d, under conditions common in ecological meta‐analyses. We simulated data from experiments, computed effect sizes and their variances, and performed meta‐analyses applying three weighting schemes (inverse variance, sample size, and unweighted) for varying levels of effect size, within‐study replication, number of studies in the meta‐analysis, and among‐study variance. Unweighted analyses, and those using weights based on sample size, were close to unbiased and yielded coverages close to the nominal level of 0.95. In contrast, the inverse‐variance weighting scheme led to bias and low coverage, especially for meta‐analyses based on studies with low replication. This bias arose because of a correlation between the estimated effect and its weight when using the inverse‐variance method. In many cases, the sample size weighting scheme was most efficient, and, when not, the differences in efficiency among the three methods were relatively minor. Thus, if using Hedges’ d, we recommend using weights based upon sample size that do not involve individual study estimates of the effect size.

show abstract

Section: Discussionmentioning

confidence: 99%

Bias in meta‐analyses using Hedges’ d

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Differences between weighted and unweighted statistics are generally small for meta‐analysis (Cardinale et al ; Marvier et al ; Benayas et al ). Furthermore, unweighted meta‐regression is often more robust because it does not use potentially misleading estimation of error variances (Fletcher & Dixon ).…”

Section: Methodsmentioning

confidence: 99%

A meta‐analysis of functional group responses to forest recovery outside of the tropics

Spake

Ezard

Martin

et al. 2015

Conservation Biology

View full text Add to dashboard Cite

Both active and passive forest restoration schemes are used in degraded landscapes across the world to enhance biodiversity and ecosystem service provision. Restoration is increasingly also being implemented in biodiversity offset schemes as compensation for loss of natural habitat to anthropogenic development. This has raised concerns about the value of replacing old‐growth forest with plantations, motivating research on biodiversity recovery as forest stands age. Functional diversity is now advocated as a key metric for restoration success, yet it has received little analytical attention to date. We conducted a meta‐analysis of 90 studies that measured differences in species richness for functional groups of fungi, lichens, and beetles between old‐growth control and planted or secondary treatment forests in temperate, boreal, and Mediterranean regions. We identified functional‐group–specific relationships in the response of species richness to stand age after forest disturbance. Ectomycorrhizal fungi averaged 90 years for recovery to old‐growth values (between 45 years and unrecoverable at 95% prediction limits), and epiphytic lichens took 180 years to reach 90% of old‐growth values (between 140 years and never for recovery to old‐growth values at 95% prediction limits). Non‐saproxylic beetle richness, in contrast, decreased as stand age of broadleaved forests increased. The slow recovery by some functional groups essential to ecosystem functioning makes old‐growth forest an effectively irreplaceable biodiversity resource that should be exempt from biodiversity offsetting initiatives.

show abstract

“…These meta‐estimates depend most strongly on study variance, which co‐varies with grain and replication, obscuring the scale dependence in effect size. Fletcher and Dixon (2012) found that unweighted meta‐regressions outperformed weighted meta‐regressions, because the former did not make use of potentially misleading information on precision, except when precision covaries with leverage in the regression. Although we cannot know the true effect size and sampling variance for the empirical studies, this covariation may contribute to the observed differences in detectability of a grain‐size influence in our empirical meta‐regressions.…”

Section: Discussionmentioning

confidence: 99%

Implications of scale dependence for cross‐study syntheses of biodiversity differences

Spake¹,

Mori

Beckmann

et al. 2020

Ecology Letters

View full text Add to dashboard Cite

Biodiversity studies are sensitive to well‐recognised temporal and spatial scale dependencies. Cross‐study syntheses may inflate these influences by collating studies that vary widely in the numbers and sizes of sampling plots. Here we evaluate sources of inaccuracy and imprecision in study‐level and cross‐study estimates of biodiversity differences, caused by within‐study grain and sample sizes, biodiversity measure, and choice of effect‐size metric. Samples from simulated communities of old‐growth and secondary forests demonstrated influences of all these parameters on the accuracy and precision of cross‐study effect sizes. In cross‐study synthesis by formal meta‐analysis, the metric of log response ratio applied to measures of species richness yielded better accuracy than the commonly used Hedges' g metric on species density, which dangerously combined higher precision with persistent bias. Full‐data analyses of the raw plot‐scale data using multilevel models were also susceptible to scale‐dependent bias. We demonstrate the challenge of detecting scale dependence in cross‐study synthesis, due to ubiquitous covariation between replication, variance and plot size. We propose solutions for diagnosing and minimising bias. We urge that empirical studies publish raw data to allow evaluation of covariation in cross‐study syntheses, and we recommend against using Hedges' g in biodiversity meta‐analyses.

show abstract

Modelling data from different sites, times or studies: weighted vs. unweighted regression

Cited by 17 publications

References 24 publications

Bias in meta‐analyses using Hedges’ d

Bias in meta‐analyses using Hedges’ d

A meta‐analysis of functional group responses to forest recovery outside of the tropics

Implications of scale dependence for cross‐study syntheses of biodiversity differences

Contact Info

Product

Resources

About