Modelling publication bias in meta-analysis: a review

Abstract

Meta-analysis is now a widely used technique for summarizing evidence from multiple studies. Publication bias, the bias induced by the fact that research with statistically significant results is potentially more likely to be submitted and published than work with null or non-significant results, poses a threat to the validity of such analyses. The implication of this is that combining only the identified published studies uncritically may lead to an incorrect, usually over optimistic, conclusion. How publication bias should be addressed when carrying out a meta-analysis is currently a hotly debated subject. While statistical methods to test for its presence are starting be used, they do not address the problem of how to proceed if publication bias is suspected. This paper provides a review of methods, which can be employed as a sensitivity analysis to assess the likely impact of publication bias on a meta-analysis. It is hoped that this will raise awareness of such methods, and promote their use and development, as well as provide an agenda for future research.

Dissemination and publication of research findings: an updated review of related biases

“…Many of the sophisticated modelling methods were discussed in depth in the 2000 HTA report 2 and in a review article by Sutton et al 483 These methods are usually based on weighted distribution theory derived from both classical 338,[484][485][486][487][488][489][490][491][492] or Bayesian [493][494][495][496][497] perspectives. There are two aspects to the selection models that use weighted distribution theory: an effect size model, which specifies what the distribution of the effect size estimate would be if there were no selection, and the selection model, which specifies how this effect size distribution is modified by the selection process.…”

Publication Bias in Union-Productivity Research?

“…Le biais de publication est un phénomène qui désigne un ensemble de distorsions dans le processus de publication des résultats de travaux de recherche (Sutton et al, 2000b). Le biais de publication peut avoir plusieurs causes, liées aux auteurs des travaux et aux comités de lecture des revues.…”

Synthesizing medical evidence: systematic reviews and metaanalyses

“…109 Many formal methods for estimating the amount of publication bias in a systematic review, and its effect on the conclusions of the review, have been published. 65,257,265,295,313,326,340,352 One simple method is to calculate the Rosenthal "fail-safe N," which is the number of unpublished studies with a null effect that would need to be added to a metaanalysis to render a treatment effect estimate nonsignificant. 277 This method assumes that unpublished trials would, on average, show a null effect for the experimental treatment.…”

“…98,338 It is also possible to model the influence of some aspect of trial results, such as the treatment effect estimate or the degree of statistical significance, on publication probability, and then to use the model to adjust the metaanalysis. 64,77,340 These methods, whether simple or complex, depend on fundamental assumptions about the missing studies that are difficult to test, and only the Rosenthal method and the funnel plot have achieved anything like widespread use, although a recent survey of the quality of systematic reviews found that fewer than 10% indicated that a funnel plot had been examined. 352 Based on various models, in some studies it has been estimated that approximately 25 to 50% of metaanalyses may be affected by publication bias and that treatment effects were overestimated by up to 47% as a result.…”