The effect of publication bias on the Q test and assessment of heterogeneity.

Augusteijn, Hilde E. M.; Aert, Robbie C. M. van; Assen, Marcel A.L.M. van

doi:10.1037/met0000197

Cited by 69 publications

(76 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 In terms of the heterogeneity metric I 2 , these values of τ = 0.2 and 0.4, in combination with the specific primary sample sizes we simulated, are approximately equal to the descriptors proposed by Pigott (2012) for "medium" (I 2 = 50%) and "large" (I 2 = 75%) heterogeneity: random-effects meta-analysis of the unbiased data (no publication bias, no QRPs, aggregated over k and δ), yields an average observed I 2 of 46% (SD = 17%) when τ = 0.2 and 77% (SD = 10%) when τ = 0.4. 3 One should keep in mind that the estimates of τ reported by van Erp et al van Erp et al (2017) may be over-or under-estimates as a result of bias (Augusteijn et al, 2018). Publication bias.…”

Section: Simulationmentioning

confidence: 99%

Correcting for bias in psychology: A comparison of meta-analytic methods

Carter¹,

Schönbrodt²,

Gervais³

et al. 2017

Preprint

View full text Add to dashboard Cite

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, much of this work has not been tailored specifically to psychology, so it is not clear which methods work best for data typically seen in our field. Here, we present a comprehensive simulation study to examine how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We created such scenarios by simulating several levels of questionable research practices, publication bias, heterogeneity, and using study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change based on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts on improving the primary literature and conducting large-scale, pre-registered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/.

show abstract

Section: Simulationmentioning

confidence: 99%

Correcting for bias in psychology: A comparison of meta-analytic methods

Carter¹,

Schönbrodt²,

Gervais³

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…These estimates are notoriously imprecise, unless the set of studies is large (Viechtbauer, 2007). Additionally, estimates of τ can and will also be systematically distorted by publication bias (Augusteijn, van Aert & van Assen, 2019;Jackson, 2007; see also Figure S16). Given the ubiquity of heterogeneity in psychological meta-analyses (Stanley, Carter & Doucouliagos, 2018), we therefore warn against the use of TES in sets of more than 10 studies.…”

Section: Type I Error Ratementioning

confidence: 99%

“…Furthermore, it can also not be estimated properly in the presence of publication bias. To demonstrate this, Augusteijn, van Aert & van Assen (2019) showed analytically that publication bias can have severe and rather complex effects on heterogeneity estimates: Depending on the combination of the actual degree of heterogeneity, true effect size and degree of censorship heterogeneity may either be under-or overestimated. Within the parameter settings of our study, the actual τ was oftentimes underestimated drastically in all four selection conditions realized (see Figure S16).…”

Section: Four Recommendations For the Application Of Tests For Publicmentioning

confidence: 99%

How to detect publication bias in psychological research? A comparative evaluation of six statistical methods

Renkewitz¹,

Keiner²

2018

Preprint

View full text Add to dashboard Cite

Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates observed in the social sciences. Both of these problems do not only increase the proportion of false positives in the literature but can also lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect and correct such bias in meta-analytic results. We present an evaluation of the performance of six of these tools in detecting bias. To assess the Type I error rate and the statistical power of these tools we simulated a large variety of literatures that differed with regard to underlying true effect size, heterogeneity, number of available primary studies and variation of sample sizes in these primary studies. Furthermore, simulated primary studies were subjected to different degrees of publication bias. Our results show that the power of the detection methods follows a complex pattern. Across all simulated conditions, no method consistently outperformed all others. Hence, choosing an optimal method would require knowledge about parameters (e.g., true effect size, heterogeneity) that meta-analysts cannot have. Additionally, all methods performed badly when true effect sizes were heterogeneous or primary studies had a small chance of being published irrespective of their results. This suggests, that in many actual meta-analyses in psychology bias will remain undiscovered no matter which detection method is used.

show abstract

“…Then the subgroup analysis, the meta-regression analysis and the sensitivity analysis were conducted to explore the potential sources of heterogeneity [14]. Finally, Begg's funnel plot and Egger's tests were applied to assess the included studies for the possibility of publication bias [15]. All statistical tests in the meta-analysis part were carried out using STATA 12.0 software.…”

Section: Statistical Methods For Data Synthesismentioning

confidence: 99%

Integrated characterization and validation of the prognostic significance of microRNA-200s in colorectal cancer

Peng

Cheng

et al. 2020

Cancer Cell Int

View full text Add to dashboard Cite

Background: Accumulating evidence has demonstrated that could serve as promising molecular biomarkers for cancer prognosis. Nevertheless, the associations between miR-200s expression and colorectal cancer (CRC) prognosis remain controversial. Methods:We applied two mainstream approaches combining meta-analysis and bioinformatics analysis to answer whether miR-200s were associated with the prognosis of CRC patients and why miR-200s could be used as prognostic biomarkers for CRC. Results:Consequently, low expression of miR-200s was associated with unfavorable overall survival (OS) in CRC patients (HR: 1.09; 95% CI 1.01-1.17; P = 0.025). According to the subgroup analysis, the prognostic role of miR-200s was more significant for tissue samples, large samples, American patients and miR-200a subgroups. Then the target genes of miR-200s were predicted and applied for functional enrichment analyses. The results showed that the target genes of miR-200s were mainly enriched into some vital ontology subjects such as regulation ability, key cell structures and binding function. Moreover, a series of important signaling pathways were identified, which were significantly linked with the initiation and progression of CRC. Additionally, a protein-protein interaction (PPI) network of miR-200s targets was constructed to screen hub genes and modules. The identified hub genes and modules were validated to be highly involved in the occurrence and development of CRC.Conclusions: Current evidences revealed that miR-200s could be promising biomarkers for CRC prognosis. However, the findings still need to be validated with more larger-scale prospective studies and biological experiments before miR-200s could be applied into clinical application.

show abstract

The effect of publication bias on the Q test and assessment of heterogeneity.

Abstract: The effect of publication bias on the assessment of heterogeneity. Psychological Methods.Feedback, suggestion, comments and remarks are more than welcome. They can be send to h.e.

Cited by 69 publications

References 67 publications

Correcting for bias in psychology: A comparison of meta-analytic methods

Correcting for bias in psychology: A comparison of meta-analytic methods

How to detect publication bias in psychological research? A comparative evaluation of six statistical methods

Integrated characterization and validation of the prognostic significance of microRNA-200s in colorectal cancer

Contact Info

Product

Resources

About