Correct standard errors can bias meta‐analysis

The partial correlation coefficient (PCC) is used to quantify the linear relationship between two variables while taking into account/controlling for other variables. Researchers frequently synthesize PCCs in a meta‐analysis, but two of the assumptions of the common equal‐effect and random‐effects meta‐analysis model are by definition violated. First, the sampling variance of the PCC cannot assumed to be known, because the sampling variance is a function of the PCC. Second, the sampling distribution of each primary study's PCC is not normal since PCCs are bounded between −1 and 1. I advocate applying the Fisher's z transformation analogous to applying Fisher's z transformation for Pearson correlation coefficients, because the Fisher's z transformed PCC is independent of the sampling variance and its sampling distribution more closely follows a normal distribution. Reproducing a simulation study by Stanley and Doucouliagos and adding meta‐analyses based on Fisher's z transformed PCCs shows that the meta‐analysis based on Fisher's z transformed PCCs had lower bias and root mean square error than meta‐analyzing PCCs. Hence, meta‐analyzing Fisher's z transformed PCCs is a viable alternative to meta‐analyzing PCCs, and I recommend to accompany any meta‐analysis based on PCCs with one using Fisher's z transformed PCCs to assess the robustness of the results.

Section: Reproducing Simulation Study By Stanley and Doucouliagosmentioning

confidence: 99%

“…Stanley and Doucouliagos 5 (henceforth S&D) conducted a simulation study to examine the effect of including

s_{1}^{2}

and

s_{2}^{2}

as part of the weights in the random‐effects meta‐analysis model. The conclusion of their simulation study is in the title of their paper: “Correct SEs can bias meta‐analysis,” so using

s_{1}^{2}

yielded more biased results in the meta‐analysis than

s_{2}^{2}

.…”

Section: Introductionmentioning

confidence: 99%

“…The conclusion of their simulation study is in the title of their paper: “Correct SEs can bias meta‐analysis,” so using

s_{1}^{2}

yielded more biased results in the meta‐analysis than

s_{2}^{2}

. S&D 5 attribute this larger bias to larger differences in the inverse variance weights when

s_{1}^{2}

is used compared to

s_{2}^{2}

. That is, the primary studies in the meta‐analysis get more equal weights when

s_{2}^{2}

is used compared to

s_{1}^{2}

.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Meta‐analyzing partial correlation coefficients using Fisher's z transformation

Aert

2023

Meta‐analysis and partial correlation coefficients: A matter of weights

Hong,

Reed

2023

This study builds on the simulation framework of a recent paper by Stanley and Doucouliagos (Research Synthesis Methods 2023;14;515–519). S&D use simulations to make the argument that meta‐analyses using partial correlation coefficients (PCCs) should employ a “suboptimal” estimator of the PCC standard error when constructing weights for fixed effect and random effects estimation. We address concerns that their simulations and subsequent recommendation may give meta‐analysts a misleading impression. While the estimator they promote dominates the “correct” formula in their Monte Carlo framework, there are other estimators that perform even better. We conclude that more research is needed before best practice recommendations can be made for meta‐analyses with PCCs.

Meta‐analyses of partial correlations are biased: Detection and solutions

Stanley,

Doucouliagos,

Havranek

2024

Self Cite

We demonstrate that all meta‐analyses of partial correlations are biased, and yet hundreds of meta‐analyses of partial correlation coefficients (PCCs) are conducted each year widely across economics, business, education, psychology, and medical research. To address these biases, we offer a new weighted average, UWLS+3. UWLS+3 is the unrestricted weighted least squares weighted average that makes an adjustment to the degrees of freedom that are used to calculate partial correlations and, by doing so, renders trivial any remaining meta‐analysis bias. Our simulations also reveal that these meta‐analysis biases are small‐sample biases (n < 200), and a simple correction factor of (n − 2)/(n − 1) greatly reduces these small‐sample biases along with Fisher's z. In many applications where primary studies typically have hundreds or more observations, partial correlations can be meta‐analyzed in standard ways with only negligible bias. However, in other fields in the social and the medical sciences that are dominated by small samples, these meta‐analysis biases are easily avoidable by our proposed methods.