Testing for baseline differences in randomized controlled trials: an unhealthy research behavior that is hard to eradicate

Abstract

BackgroundAccording to the CONSORT statement, significance testing of baseline differences in randomized controlled trials should not be performed. In fact, this practice has been discouraged by numerous authors throughout the last forty years. During that time span, reporting of baseline differences has substantially decreased in the leading general medical journals. Our own experience in the field of nutrition behavior research however, is that co-authors, reviewers and even editors are still very persistent in their demand for these tests. The aim of this paper is therefore to negate this demand by providing clear evidence as to why testing for baseline differences between intervention groups statistically is superfluous and why such results should not be published.DiscussionTesting for baseline differences is often propagated because of the belief that it shows whether randomization was successful and it identifies real or important differences between treatment arms that should be accounted for in the statistical analyses. Especially the latter argument is flawed, because it ignores the fact that the prognostic strength of a variable is also important when the interest is in adjustment for confounding. In addition, including prognostic variables as covariates can increase the precision of the effect estimate. This means that choosing covariates based on significance tests for baseline differences might lead to omissions of important covariates and, less importantly, to inclusion of irrelevant covariates in the analysis. We used data from four supermarket trials on the effects of pricing strategies on fruit and vegetables purchases, to show that results from fully adjusted analyses sometimes do appreciably differ from results from analyses adjusted for significant baseline differences only. We propose to adjust for known or anticipated important prognostic variables. These could or should be pre-specified in trial protocols. Subsequently, authors should report results from the fully adjusted as well as crude analyses, especially for dichotomous and time to event data.SummaryBased on our arguments, which were illustrated by our findings, we propose that journals in and outside the field of nutrition behavior actively adopt the CONSORT 2010 statement on this topic by not publishing significance tests for baseline differences anymore.

Randomized controlled trial comparing operative times between standard and robot‐assisted laparoscopic hysterectomy

“…However, as a randomized trial any baseline differences were due to chance and, consequently, the primary analysis did not include any baseline characteristics as covariates [12]. …”

Delivery and Receipt of a Self-Determination-Theory-Based Extracurricular Physical Activity Intervention: Exploring Theoretical Fidelity in Action 3:30

“…Initially, unadjusted models were estimated, and then models were adjusted for baseline outcome variable score, BMI, and deprivation. Both unadjusted and adjusted models are presented to facilitate the comparison of our results with results from other trials that are different but comparable, especially where different covariates have been included; this practice is in line with recent recommendations (de Boer, Waterlander, Kuijper, Steenhuis, & Twisk, 2015). As described by Peters, Richards, Bankhead, Ades, and Sterne (2003), robust standard errors were estimated to account for the clustering of pupils within schools.…”

Statistical testing of baseline differences in sports medicine RCTs: a systematic evaluation

“…2 As shown in figure 1, our PubMed search of RCTs retrieved 1109 articles that were then filtered using keywords to remove obvious systematic reviews, other non-RCTs and crossover trials to develop a set of 598 potential articles to evaluate. Then, two reviewers (RLP and MT) independently examined each article and came to a consensus on which articles were RCTs, which contained tables of baseline differences between randomised groups, and which had significance testing of baseline differences.…”