Spurious Precision in Meta-Analysis of Observational Research

Iršová, Zuzana; Bom, Pedro R. D.; Havránek, Tomáš; Rachinger, Heiko

doi:10.31222/osf.io/3qp2w

Cited by 13 publications

(12 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, where b ij denotes the i-th beauty effect estimated in the j-th study, and SE(b ij ) denotes its standard error. FE = study-level fixed effects, BE = study-level between effects, MAIVE = Meta-Analysis Instrumental Variable Estimator (Irsova et al, 2023) with the inverse of the square root of the sample size used as an instrument for the standard error. Weighted = the inverse of the number of estimates per study is used as the weight.…”

Section: Sample Without Prostitutes Prostitutesmentioning

confidence: 99%

“…Precision can be p-hacked (changed by changing specification in order to achieve statistical significance), which introduces reverse causality. To address these problems we use the novel estimator due to Irsova et al (2023), which accounts for the potential endogeneity of the standard error and various forms of phacking. We also use recently developed nonlinear techniques by Andrews & Kasy (2019), Bom & Rachinger (2019, Furukawa (2020), andIoannidis et al (2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Beauty and Professional Success: A Meta-Analysis

Bortnikova,

Havranek,

Irsova

2024

Preprint

View full text Add to dashboard Cite

Common wisdom suggests that beauty helps in the labor market. We show that two factors combine to explain away the mean beauty premium reported in the literature. First, correcting for publication bias reduces the premium by at least a third. Second, controlling for cognitive ability negates the premium for all occupations except sex workers, a point further underscored by the similarity of the beauty effect on earnings and productivity. The second factor implies a positive link, perhaps genetic, between beauty and intelligence. We find little evidence of substantial attenuation bias that could offset publication and omitted-variable biases. The empirical literature is inconsistent with discrimination based solely on tastes for beauty. To obtain these results we collect 1,159 estimates of the effect of beauty on earnings or productivity reported in 67 studies and codify 33 aspects that reflect estimation context, including the potential intensity of attenuation bias. We employ recently developed techniques to account for publication bias and model uncertainty.

show abstract

Section: Sample Without Prostitutes Prostitutesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Beauty and Professional Success: A Meta-Analysis

Bortnikova,

Havranek,

Irsova

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…5 4 See Borenstein (2009) for a discussion of bunching with increasing block tariffs. 5 Ring 1 covers the area within 25 km from Tbilisi; ring 2 that between 25 and 50 km; ring 3 that between 50 and 100 km; ring 4 that between 100 and 150 km; ring 4 that from 150 to 200 km; ring 6 that from 200 to 250 km, and ring 7 that between 250 and 300 km.…”

Section: What Effects Do We Expect From the Policy?mentioning

confidence: 99%

Heterogeneous Effects of Government Energy Assistance Programs:Covid-19 Lockdowns in the Republic of Georgia

Alberini,

Bezhanishvili,

anon

2024

Preprint

View full text Add to dashboard Cite

“…Another alternative solution, which we do not simulate here, would be to use the square root of the inverse sample size as an instrument for the standard error in PET-PEESE. [31][32] Among other things, the instrumental-variable PET-PEESE technique accounts for the mechanical correlation between r and its SE. See Irsova, Bom, Havranek, and Rachinger 32 for simulations of this instrumental-variable approach.…”

Section: Pet-peese Model Of Publication Selection Biasmentioning

confidence: 99%

Reducing the biases of the conventional meta-analysis of correlations

Stanley,

Doucouliagos,

Havranek

2023

Preprint

View full text Add to dashboard Cite

Conventional meta-analyses of correlations are biased due to the correlation between the estimated correlation and its standard error. Simulations that are closely calibrated to match actual research conditions widely seen across correlational studies in psychology corroborate these biases and suggest a solution. UWLS+3 is a simple inverse-variance weighted average (the unrestricted weighted least squares) that adjusts the degrees of freedom and thereby reduces small-sample bias to scientific negligibility. UWLS+3 is also less biased than conventional random-effects estimates of correlations and Fisher’s z, whether or not there is publication selection bias. However, publication selection bias remains a ubiquitous source of bias and false positive findings. Despite the correlation between the estimated correlation and its standard error even in the absence of any selective reporting, the precision-effect test/precision-effect estimate with standard error (PET-PEESE) nearly eradicates publication selection bias. PET-PEESE keeps the rate of false positives (i.e., type I errors) within their nominal levels under the typical conditions widely seen across psychological research and with or without publication selection bias.

show abstract

Spurious Precision in Meta-Analysis of Observational Research

Cited by 13 publications

References 87 publications

Beauty and Professional Success: A Meta-Analysis

Beauty and Professional Success: A Meta-Analysis

Heterogeneous Effects of Government Energy Assistance Programs:Covid-19 Lockdowns in the Republic of Georgia

Reducing the biases of the conventional meta-analysis of correlations

Contact Info

Product

Resources

About