Publication Bias in Meta‐Analysis

Abstract

Publication bias is the term for what occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies. Simply put, when the research that is readily available differs in its results from the results of all the research that has been done in an area, readers and reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows. In some cases this can have dramatic consequences, as when an ineffective or dangerous treatment is falsely viewed as safe and effective. This can be illustrated by two events that received much media attention as this book was going to press in late 2004. These are the debate surrounding Merck's recall of Vioxx, a popular arthritis drug (Merck maintained that it recalled Vioxx as soon as the data indicated the high prevalence of cardiovascular events among those who took Vioxx for more than 18 months, while media reports said that Merck hid adverse event evidence for years), and the use of selective serotonin reuptake inhibitor (SSRI) anti-depressants among adolescents (Elliott Spitzer, attorney general of New York State, filed a 2004 lawsuit against GlaxoSmithKline,

Publication Bias in the Organizational Sciences

“…The same holds true for unstructured interviews. We thus conclude that the effect of publication bias in McDaniel et al (1994) overall is minimal (McDaniel et al, 2006;Rothstein et al, 2005a). However, there appear to be substantial differences between distributions involving samples published in journal articles and samples from other sources, indicating that sample suppression may have taken place; that is, only samples with significant effect sizes tend to get published in our journals.…”

“…Similarly, article rejections during the editorial review process contribute to sample-level causes of publication bias, particularly if the samples used in the rejected article never become publically available. Reasons for this may include poorly framed studies, small sample size studies, statistically insignificant findings, and results contrary to conventional wisdom, theory, or trends of past research (Banks & McDaniel, 2011;Chan et al, 2004;Dickersin, 2005;Pigott, 2009;Rothstein et al, 2005a;Sutton & Pigott, 2005). Thus, authors, reviewers, and editors can prevent the publication of results from an entire sample.…”

“…641 (Chan et al, 2004;Dickersin, 2005;McDaniel et al, 2006;Pigott, 2009;Rothstein et al, 2005a). However, the cumulative point estimate stabilizes relatively quickly (e.g., the cumulative point estimate after the first seven samples is almost identical to the meta-analytic estimate based on all samples), indicating that publication bias may not be present.…”

Lose biodiversity, gain disease

“…Third, metaanalyses can only report on published papers. A common criticism is that of publication bias: significant results are more likely to be published (18). A more substantial issue here (1) is how additional species were selected by the generators of the primary research.…”

Publication bias in the anesthesiology literature: shifting the focus from the “positive” to the “truth”

“…Publication bias, defined as ''published research which is systematically unrepresentative of the population of completed studies'', 1 and referred to as the ''file drawer'' 2 problem as early as 1979, is certainly not a new issue. 3 In practice, publication bias usually refers to selective publication of clinical studies with at least one statistically significant difference in important study outcomes (socalled positive studies) and selective non-publication of studies with no differences between groups (so-called negative studies).…”