The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses

Abstract

Publication bias is a type of systematic error when synthesizing evidence that cannot represent the underlying truth. Clinical studies with favorable results are more likely published and thus exaggerate the synthesized evidence in meta-analyses. The trim-and-fill method is a popular tool to detect and adjust for publication bias. Simulation studies have been performed to assess this method, but they may not fully represent realistic settings about publication bias. Based on real-world meta-analyses, this article provides practical guidelines and recommendations for using the trim-and-fill method. We used a worked illustrative example to demonstrate the idea of the trim-and-fill method, and we reviewed three estimators ( R 0 , L 0 , and Q 0 ) for imputing missing studies. A resampling method was proposed to calculate P values for all 3 estimators. We also summarized available meta-analysis software programs for implementing the trim-and-fill method. Moreover, we applied the method to 29,932 meta-analyses from the Cochrane Database of Systematic Reviews , and empirically evaluated its overall performance. We carefully explored potential issues occurred in our analysis. The estimators L 0 and Q 0 detected at least one missing study in more meta-analyses than R 0 , while Q 0 often imputed more missing studies than L 0 . After adding imputed missing studies, the significance of heterogeneity and overall effect sizes changed in many meta-analyses. All estimators generally converged fast. However, L 0 and Q 0 failed to converge in a few meta-analyses that contained studies with identical effect sizes. Also, P values produced by different estimators could yield different conclusions of publication bias significance. Outliers and the pre-specified direction of missing studies could have influential impact on the trim-and-fill results. Meta-analysts are recommended to perform the trim-and-fill method with great caution when using meta-analysis software programs. Some default settings (e.g., the choice of estimators and the direction of missing studies) in the programs may not be optimal for a certain meta-analysis; they should be determined on a case-by-case basis. Sensitivity analyses are encouraged to examine effects of different estimators and outlying studies. Also, the trim-and-fill estimator should be routinely reported in meta-analyses, because the results depend highly on it.

Extent of Primary DNA Damage Measured by the Comet Assay in Health Professionals Exposed to Antineoplastic Drugs: A Systematic Review and Meta-Analysis

“…The trim and fill method is a statistical approach aimed to estimate potential missing studies, causing the asymmetry of the funnel plot. This method assumes that the studies with the most extreme ES have to be suppressed, adjusting the overall effect estimate [25]. To perform the meta-analysis, the software Prometa3 ® (Internovi, Italy) was used.…”

Pre-pregnancy Maternal Weight and Gestational Weight Gain Increase the Risk for Childhood Asthma and Wheeze: An Updated Meta-Analysis

“…The symmetry of funnel plots was appraised by the Egger's test. The trim-and-fill method was used to estimate the number of potentially missing studies stemming from publication bias (21). Significant publication bias was recorded if the probability of the Egger's test is <10%.…”

Efficacy and Safety of Transurethral Laser Surgery Versus Transurethral Resection for Non-Muscle-Invasive Bladder Cancer: A Meta-Analysis and Systematic Review

“…4). Further verification by the trim-and-fill method showed that, although publication bias may exist, our results were robust [35].…”