Recommendations on multiple testing adjustment in multi-arm trials with a shared control group

Abstract

Multi-arm clinical trials assessing multiple experimental treatments against a shared control group can offer efficiency advantages over independent trials through assessing an increased number of hypotheses. Published opinion is divided on the requirement for multiple testing adjustment to control the familywise type-I error rate (FWER). The probability of a false positive error in multiarm trials compared to equivalent independent trials is affected by the correlation between comparisons due to sharing control data. We demonstrate that this correlation in fact leads to a reduction in the FWER, therefore FWER adjustment is not recommended solely due to sharing control data. In contrast, the correlation increases the probability of multiple false positive outcomes across the hypotheses, although standard FWER adjustment methods do not control for this. A stringent critical value adjustment is proposed to maintain equivalent evidence of superiority in two correlated comparisons to that obtained within independent trials. FWER adjustment is only required if there is an increased chance of making a single claim of effectiveness by testing multiple hypotheses; not due to sharing control data. For competing experimental therapies, the correlation between comparisons can be advantageous as it eliminates bias due to the experimental therapies being compared to different control populations.

Dressings and securements for the prevention of peripheral intravenous catheter failure in adults (SAVE): a pragmatic, randomised controlled, superiority trial

“…Alpha adjustment was not undertaken consistent with our study design of separate hypothesis testing of the effect of multiple alternative treatments (that did not inform a single claim of effectiveness) on one primary endpoint, with a shared control group. [22] This resulted in each group requiring 388 participants, plus 10% for potential attrition (1708 in total).…”

“…We acknowledge some published views differ, but we did not undertake alpha adjustment as our study design tested separate hypotheses of three alternative treatments (not varying doses of the same treatment) for one primary endpoint against a shared control group. [22,30,31] Both the Centers for Disease Control Guidelines and the Infusion Therapy Standards of Practice consider optimal PIVC dressing and securement to be unresolved. [10,13] Both documents reflect the conclusions of a 2015 systematic review and meta-analysis which highlighted the paucity of high quality randomised studies.…”

Current Statistical Considerations and Regulatory Perspectives on the Planning of Confirmatory Basket, Umbrella, and Platform Trials

“…This correlation will not further increase the probability of having at least one false positive rejection within the trial as compared with the situation of independent trials with a dedicated control group for each comparison. On the contrary, using a common control group will lead to a lower probability of at least one false positive rejection within the trial …”

“…In this example, this is thus the chance of simultaneous multiple false positive regulatory decisions, rather than the type I error rate per se , which is inflated. The former indeed increases when sharing controls as compared with the standard setting of separate clinical trials . In order to reduce the chance of multiple simultaneous false positive decisions to what would have been obtained with separate trials, lower individual values of the type I error rate can be used for each comparison, or more patients can be randomized to the control group .…”

A web application for the design of multi-arm clinical trials

“…Yet, if a series of two-arm trials were conducted, no adjustment would be made to the significance level used in each trial. For brevity, we will not repeat all further arguments on this issue here, and instead refer the reader to several key discussions on multiplicity [5,8,9,10,11,12,13,14,15,16,17,18].…”