The evidence provided by a single trial is less reliable than its statistical analysis suggests

Small studies are more heterogeneous than large ones: a meta-meta-analysis

“…Especially in randomeffects models and with substantial heterogeneity, the influence of small studies will be major and may affect the reliability of meta-analyses. On the other hand, simulations have shown that a meta-analysis containing many, possibly small, studies is better than a single large trial able to estimate the treatment effect [6,13,14], even when there is some publication bias [15]. Roloff et al [16] showed that in case of cumulative meta-analysis, it is more powerful to add several small studies than one or a few large studies because the between-study heterogeneity can be estimated more precisely when more studies, either small or large, are available.…”

Hypothermia after cardiac arrest should be further evaluated—A systematic review of randomised trials with meta-analysis and trial sequential analysis

“…Contrary to other reviews of MIH, we have evaluated systematic and random errors and incorporated the GRADE classification, thus broadening the base for a well-founded judgement of the evidence. The risk of type-I errors has not previously been evaluated in this field, but growing evidence highlights this as one of the major problems of spurious findings due to meta-analyses [48,69] and may therefore provide a valuable addition.…”

Evidence‐based critical care medicine: seeing through a glass darkly

“…Caution has been advised in basing clinical practice on the findings of single-centre trials [49]. Such studies commonly lack scientific rigour and may not be generalisable to other units and situations, and there are statistical reasons why these trials may not be as reliable [50]. The management of critically ill patients is complex, often involving multiple interventions and processes, and it is often not realistic or possible to isolate single interventions from the totality of care.…”