Investigating and dealing with publication bias and other reporting biases in meta‐analyses of health research: A review

Abstract

A P value, or the magnitude or direction of results can influence decisions about whether, when, and how research findings are disseminated. Regardless of whether an entire study or a particular study result is unavailable because investigators considered the results to be unfavorable, bias in a meta‐analysis may occur when available results differ systematically from missing results. In this article, we summarize the empirical evidence for various reporting biases that lead to study results being unavailable for inclusion in systematic reviews, with a focus on health research. These biases include publication bias and selective nonreporting bias. We describe processes that systematic reviewers can use to minimize the risk of bias due to missing results in meta‐analyses of health research, such as comprehensive searches and prospective approaches to meta‐analysis. We also outline methods that have been designed for assessing risk of bias due to missing results in meta‐analyses of health research, including using tools to assess selective nonreporting of results, ascertaining qualitative signals that suggest not all studies were identified, and generating funnel plots to identify small‐study effects, one cause of which is reporting bias. Highlights Bias in a meta‐analysis may occur when available results differ systematically from missing results. Several different tools, plots, and statistical methods have been designed for assessing risk of bias due to missing results in meta‐analyses. These include comparison of prespecified analysis plans with completed reports to detect selective nonreporting of results, consideration of qualitative signals that suggest not all studies were identified, and the use of funnel plots to identify small‐study effects, for which reporting bias is one of several causes. Information from approaches such as funnel plots and selection models is more difficult to interpret than from less subjective approaches such as detection of incompletely reported results in studies for which prespecified analysis plans were available.

The Relationship of Tree Nuts and Peanuts with Adiposity Parameters: A Systematic Review and Network Meta-Analysis

“…To determine the potential clinical effects of nut consumption in people with overweight or obesity and in healthy participants, we conducted analyses based on the percentage of change, considering 5% of change in each baseline outcome a clinically significant difference and also taking into account the length of the intervention (<12 weeks vs. ≥12 weeks). Finally, we performed sensitivity analyses to assess the robustness of the summary estimates, and we assessed the publication bias and the small-study effect by a network funnel plot to determine the principle of symmetry [ 39 ]. Analyses were conducted in Stata 15.0 (Stata, College Station, TX, USA).…”

The current phase of social sciences research: A thematic overview of the literature

“…Making negative findings available is essential to the interpretation of the overall significance of a field of research. Underreporting and publication of negative results bring bias into meta-analysis and mar the validity of its application (Page et al, 2020). An essential feature of science is self-correction.…”

The Efficacy and Safety of Carbon Ion Radiotherapy for Meningiomas: A Systematic Review and Meta-Analysis

“…Funnel plots illustrate the effect sizes of primary studies as a function of study precision. Asymmetry in plots can indicate publication bias ( 24 ). Egger’s regression test yields a statistical verification of funnel plot asymmetry.…”