Multivariate Meta-Analysis: The Effect of Ignoring Within-Study Correlation

Riley, Richard D

doi:10.1111/j.1467-985x.2008.00593.x

Cited by 187 publications

(311 citation statements)

References 67 publications

Supporting

Mentioning

296

Contrasting

Unclassified

Order By: Relevance

“…Multivariate meta-analysis models are increasingly used to synthesise multiple correlated outcomes in meta-analysis [11], to improve the efficiency of summary estimates by borrowing strength across outcomes and thereby reducing the impact of missing data [12,36]. In this work we used the multivariate normal approximation to the multinomial distribution to model the observed data in each study.…”

Section: Discussionmentioning

confidence: 99%

“…In particular, test accuracy results from neighbouring thresholds are likely to be quite similar, and results for a missing threshold are bounded between any pair of higher and lower thresholds that are available. Statistically it is appealing to utilise such related information and, if possible, 'borrow strength' by considering all thresholds simultaneously [11][12][13][14]. 25 1In this article we propose a new approach for meta-analysing diagnostic test accuracy studies when there are multiple threshold results per study, when the studies use the same (or similarly validated or standardised) methods of measuring the continuous test (e.g.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Meta-Analysis of Test Accuracy Studies with Multiple and Missing Thresholds: A Multivariate-Normal Model

Riley¹,

Takwoingi²,

Trikalinos³

et al. 2014

J Biom Biostat

View full text Add to dashboard Cite

Background: When meta-analysing studies examining the diagnostic/predictive accuracy of classifications based on a continuous test, each study may provide results for one or more thresholds, which can vary across studies. Researchers typically meta-analyse each threshold independently. We consider a multivariate meta-analysis to synthesise results for all thresholds simultaneously and account for their correlation. Methods:We assume that the logit sensitivity and logit specificity estimates follow a multivariate-normal distribution within studies. We model the true logit sensitivity (logit specificity) as monotonically decreasing (increasing) functions of the continuous threshold. This produces a summary ROC curve, a summary estimate of sensitivity and specificity for each threshold, and reveals the heterogeneity in test accuracy across studies. Application is made to 13 studies of protein:creatinine ratio (PCR) for detecting significant proteinuria in pregnancy that each report up to nine thresholds, with 23 distinct thresholds across studies. Results:In the example there were large within-study and between-study correlations, which were accounted for by the method. A cubic relationship on the logit scale was a better fit for the summary ROC curve than a linear or quadratic one. Between-study heterogeneity was substantial. Based on the summary ROC curve, a PCR value of 0.30 to 0.35 corresponded to maximal pair of summary sensitivity and specificity. Limitations of the proposed model include the need to posit parametric functions for the relationship of sensitivity and specificity with the threshold, to ensure correct ordering of summary threshold results, and the multivariate-normal approximation to the within-study sampling distribution. Conclusion:The joint analysis of test performance data reported over multiple thresholds is feasible. The proposed approach handles different sets of available thresholds per study, and produces a summary ROC curve and summary results for each threshold to inform decision-making.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Meta-Analysis of Test Accuracy Studies with Multiple and Missing Thresholds: A Multivariate-Normal Model

Riley¹,

Takwoingi²,

Trikalinos³

et al. 2014

J Biom Biostat

View full text Add to dashboard Cite

show abstract

“…Values for the mean effects were chosen to be representative for the small and moderate effects commonly found in social and behavioral sciences (Cohen, 1988). The value used for the between-study variance in effect sizes (0.10) results in a realistic ratio with the sampling variance of observed effect sizes (about 0.08 for studies with n 0 25 and 0.04 for studies with n 0 50) and avoids the possibility that the effects of correlation at either level become ignorable (Riley, 2009). Covariances at the subject level were chosen to cover the range of possible values of correlation coefficients (covariances correspond with correlation coefficients of 0, .4, and .8).…”

Section: Simulation Studymentioning

confidence: 99%

Three-level meta-analysis of dependent effect sizes

et al. 2012

View full text Add to dashboard Cite

Although dependence in effect sizes is ubiquitous, commonly used meta-analytic methods assume independent effect sizes. We describe and illustrate three-level extensions of a mixed effects meta-analytic model that accounts for various sources of dependence within and across studies, because multilevel extensions of metaanalytic models still are not well known. We also present a three-level model for the common case where, within studies, multiple effect sizes are calculated using the same sample. Whereas this approach is relatively simple and does not require imputing values for the unknown sampling covariances, it has hardly been used, and its performance has not been empirically investigated. Therefore, we set up a simulation study, showing that also in this situation, a threelevel approach yields valid results: Estimates of the treatment effects and the corresponding standard errors are unbiased.Keywords Multilevel . Meta-analysis . Multiple outcomes . Dependence Three-level meta-analysis of dependent effect sizesMeta-analysis refers to the statistical integration of the quantitative results of several studies. For instance, if several experimental studies examine the effect of student coaching on students' achievement, a meta-analysis can be used to investigate how large the effect is on average, whether this effect varies over studies, and whether we can find factors that are related to the size of the effect (for an introduction to meta-analysis, see Borenstein, Hedges, Higgins, & Rothstein, 2009). Most often, the effects that were observed in the primary studies are not exactly the same as the ones predicted on the basis of the meta-analytic model. Most meta-analytic methods assume that these deviations of the observed effect sizes from their expected values are independent. This means, it is assumed that one specific observed effect size does not give information about the direction or size of deviation of another observed effect from the value we would expect on the basis of the metaanalytic model. Nevertheless, in practice, there are numerous sources of effect size dependence, and therefore, metaanalysts are very often confronted with dependent effect sizes. For instance, if several studies were done by the same research group, it is not unlikely that the effect sizes from this research group are more similar than effect sizes from two different research groups, because the effect sizes might be influenced by common factors, including the way the dependent and independent variables were operationalized, the characteristics of the studied population, the way subjects were sampled, and the observers or interviewers who collected the data (Cooper, 2009). In the same way, study results from the same country might be more similar than study results from different countries, again inducing dependence in observed effects.Besides dependence over studies, there also might be dependence within studies. For instance, a specific effect can be investigated for several samples in a single study. In this sit...

show abstract

“…However, their univariate meta-analysis approach bears an essential caveat, since the computation of mean values across primary studies does not account for interactions between the examined proxy variables. Riley (2009) shows that ignoring these dependencies in a meta-analysis can lead to a heavily biased estimation of the aggregated results. Furthermore, independent testing of correlated effects increases the chance of finding spuriously significant results (Bender et al 2008).…”

Section: Introductionmentioning

confidence: 99%

What do we really know about corporate hedging? A meta-analytical study

Geyer‐Klingeberg

Hang

Rathgeber

et al. 2017

Bus Res

View full text Add to dashboard Cite

This paper employs meta-analysis to aggregate and systematically analyze the mixed empirical evidence on the determinants of corporate hedging reported in 132 previously published studies covering data from more than 73,000 firms. Among the fourteen proxy variables analyzed by multivariate meta-analysis, three variables emerge as reliable explanatory factors for corporate hedging decisions supporting the bankruptcy and financial distress hypothesis: dividend yield (positive sign), liquidity (negative sign), and firm size (positive sign). Moreover, for tax-loss carry forwards (positive sign) and research and development (positive sign), our findings indicate a weak impact on corporate hedging behavior reflecting tax reasons, the coordination between financing and investment, and agency conflicts between shareholders and debtholders. Regarding the asymmetric information and agency conflicts of equity hypothesis, we find no explanatory power. The further analysis of heterogeneity via meta-regression reveals several factors that determine the mixed empirical evidence reported in previous studies. First, the results indicate that studies analyzing firms from North America report, on average, a lower impact of leverage on the corporate hedging decision. Moreover, studies examining more recent data samples tend to find a weaker relation between tangible assets and hedging, R&D and hedging, respectively. Overall, our results encourage scientific research to put more emphasis on finer-grained examinations of hedging variations and to discover rationales of corporate hedging extending classical financial theories.

show abstract

Multivariate Meta-Analysis: The Effect of Ignoring Within-Study Correlation

Cited by 187 publications

References 67 publications

Meta-Analysis of Test Accuracy Studies with Multiple and Missing Thresholds: A Multivariate-Normal Model

Meta-Analysis of Test Accuracy Studies with Multiple and Missing Thresholds: A Multivariate-Normal Model

Three-level meta-analysis of dependent effect sizes

What do we really know about corporate hedging? A meta-analytical study

Contact Info

Product

Resources

About