A critical reflection on computing the sampling variance of the partial correlation coefficient

Partial correlation coefficients are often used as effect sizes in the meta‐analysis and systematic review of multiple regression analysis research results. There are two well‐known formulas for the variance and thereby for the standard error (SE) of partial correlation coefficients (PCC). One is considered the “correct” variance in the sense that it better reflects the variation of the sampling distribution of partial correlation coefficients. The second is used to test whether the population PCC is zero, and it reproduces the test statistics and the p‐values of the original multiple regression coefficient that PCC is meant to represent. Simulations show that the “correct” PCC variance causes random effects to be more biased than the alternative variance formula. Meta‐analyses produced by this alternative formula statistically dominate those that use “correct” SEs. Meta‐analysts should never use the “correct” formula for partial correlations' standard errors.

“…

S_{2}^{2}

conserves the t ‐values and the p ‐values of the original reported research;

S_{1}^{2}

does not. Equation () of van Aert and Goos 4 correctly shows how

S_{2}

reproduces the same test statistics ( t ‐values and p ‐values) as the original research studies reported and used to calculate these PCCs. In contrast, PCCs with

S_{1}

misclassify the statistical significance of reported results.…”

Section: Design Bias Rmse Coverage ρ N S12 S22 S12 S22 S12 S22mentioning

confidence: 72%

“…Partial correlation coefficients,

r_{p}

S_{1}^{2} = \frac{{(1 - r_{p}^{2})}^{2}}{italicdf},

where,

df

= the degrees of freedom available to the multiple regression used to estimate

r_{p}

. Van Aert and Goos 4 claim that: “Using a suboptimal estimator of the sampling variance biases the results of meta‐analysis” (p. 4).…”

Section: Design Bias Rmse Coverage ρ N S12 S22 S12 S22 S12 S22mentioning

confidence: 99%

“…Van Aert and Goos 4 reminds meta‐analysts that the variance of the partial correlation coefficient's (PCC) sampling distribution is: 5,6

S_{1}^{2} = \frac{{(1 - r_{p}^{2})}^{2}}{italicdf},

where,

df

= the degrees of freedom available to the multiple regression used to estimate

r_{p}

S_{2}^{2}

S_{2}^{2} = \frac{((), 1 goodbreak- r_{p}^{2})}{italicdf} .

Note that the only difference in these two formulae is that the numerator of

S_{2}^{2}

is not squared as it is in

S_{1}^{2}

.…”

Section: Design Bias Rmse Coverage ρ N S12 S22 S12 S22 S12 S22mentioning

confidence: 99%

S_{2}^{2}

S_{2}^{2} = \frac{((), 1 goodbreak- r_{p}^{2})}{italicdf} .

Note that the only difference in these two formulae is that the numerator of

S_{2}^{2}

is not squared as it is in

S_{1}^{2}

. Because −1 ≤

r_{p}

, ≤1,

S_{1}^{2}

S_{2}^{2}

for all

∣ r_{p} ∣

≠ {0 or 1}.…”

Section: Design Bias Rmse Coverage ρ N S12 S22 S12 S22 S12 S22mentioning

confidence: 99%

“…However, worse still is that all meta‐analysis estimates of the mean effect where

S_{1}^{2}

is employed are biased and more biased than when

S_{2}^{2}

is alternatively utilized. Table 1 shows that the PCC variance formula,

S_{1}^{2}

, recommended by van Aert and Goos 4 yields meta‐analysis estimates that are more biased and have larger mean square error (MSE) than the alternative formula,

S_{2}^{2}

Section: Design Bias Rmse Coverage ρ N S12 S22 S12 S22 S12 S22mentioning

confidence: 99%

See 3 more Smart Citations

Correct standard errors can bias meta‐analysis

Stanley

Doucouliagos

2023

Meta‐analyzing partial correlation coefficients using Fisher's z transformation

Aert

2023

Self Cite

The partial correlation coefficient (PCC) is used to quantify the linear relationship between two variables while taking into account/controlling for other variables. Researchers frequently synthesize PCCs in a meta‐analysis, but two of the assumptions of the common equal‐effect and random‐effects meta‐analysis model are by definition violated. First, the sampling variance of the PCC cannot assumed to be known, because the sampling variance is a function of the PCC. Second, the sampling distribution of each primary study's PCC is not normal since PCCs are bounded between −1 and 1. I advocate applying the Fisher's z transformation analogous to applying Fisher's z transformation for Pearson correlation coefficients, because the Fisher's z transformed PCC is independent of the sampling variance and its sampling distribution more closely follows a normal distribution. Reproducing a simulation study by Stanley and Doucouliagos and adding meta‐analyses based on Fisher's z transformed PCCs shows that the meta‐analysis based on Fisher's z transformed PCCs had lower bias and root mean square error than meta‐analyzing PCCs. Hence, meta‐analyzing Fisher's z transformed PCCs is a viable alternative to meta‐analyzing PCCs, and I recommend to accompany any meta‐analysis based on PCCs with one using Fisher's z transformed PCCs to assess the robustness of the results.

Meta‐analysis and partial correlation coefficients: A matter of weights

Hong,

Reed

2023

This study builds on the simulation framework of a recent paper by Stanley and Doucouliagos (Research Synthesis Methods 2023;14;515–519). S&D use simulations to make the argument that meta‐analyses using partial correlation coefficients (PCCs) should employ a “suboptimal” estimator of the PCC standard error when constructing weights for fixed effect and random effects estimation. We address concerns that their simulations and subsequent recommendation may give meta‐analysts a misleading impression. While the estimator they promote dominates the “correct” formula in their Monte Carlo framework, there are other estimators that perform even better. We conclude that more research is needed before best practice recommendations can be made for meta‐analyses with PCCs.