Finding your way out of the forest without a trail of bread crumbs: development and evaluation of two novel displays of forest plots

Abstract

Research has shown that forest plots are a gold standard in the visualization of meta-analytic results. However, research on the general interpretation of forest plots and the role of researchers' meta-analysis experience and field of study is still unavailable. Additionally, the traditional display of effect sizes, confidence intervals, and weights have repeatedly been criticized. The current work presents an online statistical cognition experiment in which a total of 279 researchers with experience in meta-analysis from 36 countries evaluated conventional forest plots and two novel versions of forest plots, namely, thick forest plots and rainforest plots. The results indicate certain biases in the interpretation of forest plots, especially with regard to heterogeneity, the distribution of weights, and the theoretical concept of confidence intervals. Although the two novel displays (thick forest plots and rainforest plots) are associated with slightly longer viewing times, they are at least as well-suited and esthetically and perceptively pleasing as the conventional displays while facilitating the correct and exhaustive interpretation of the meta-analytic information. Furthermore, it is advisable to combine conventional forest plots with distribution information of the individual effects, make confidence lines more visually striking, and to display a background grid in the graph.

Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis

“…Moreover, to avoid redundancies, we did not consider variants of variants. For example, the rainforest plot is a recently proposed variant of the forest plot [23] and, as such, was added to the graph collection. However, variants of the rainforest plot (e.g., a subgroup rainforest plot) were not added to the collection, because the rainforest plot itself is already a variant, and a subgroup forest plot (as a variant of the forest plot) was already included.…”

“…As of this writing, Research Synthesis Methods is the only journal exclusively dedicated to the methodology of meta-analysis and systematic reviews, and BMC Medical Research Methodology has a long tradition in publishing methodological approaches to meta-analysis and systematic reviews in the health sciences. Both journals regularly publish papers on new data-visualization methods in this context (e.g., [22][23][24][25][26]).…”

Rainforest plots for the presentation of patient-subgroup analysis in clinical trials

“…> mod.sum<-glm(mort ~ group, family = "binomial", data = df) > rslt.sum <-rbind(rslt, data.frame(subgroup = "summary", coef = mod.sum$coef [2], SE = sqrt(diag(vcov(mod.sum))) [2]))…”

“…As a matter of fact, the likelihood of values within the interval decreases, as they approach the outer boundaries (2).…”

“…To address these shortcomings of the conventional forest plot, the rainforest plot has been proposed in the context of meta-analyses (2). In rainforest plots, each subgroup is represented by a likelihood raindrop (Figure 1).…”