Non‐inferiority trials: the ‘at least as good as’ criterion

Abstract

To demonstrate in a clinical trial that a new or experimental therapy (et) is 'at least as good as' a standard therapy (st), a statistical test or confidence interval procedure must rule out clinical inferiority with a high probability. The term 'at least as good as' implies equivalent but not necessarily superior efficacy. As it is statistically impossible to demonstrate equivalence (that is, prove the null hypothesis of no difference), Blackwelder proposed a one-sided significance test to reject the null hypothesis that standard therapy is better than experimental therapy by a clinically acceptable amount, delta(BW). In this paper, Blackwelder's approach is redefined in terms of the ratio of two means (R(True)= mu(et)/mu(st)) based on a continuous variate with higher values denoting greater improvement. The ratio-based equivalents to Blackwelder's hypotheses will be shown. The ratio parameter has the benefit of being available as a dimensionless percentage, not tied to a specified difference in means. Thus, a study can be sized to assure, with high probability, that the experimental therapy is 'at least' (R(LB)x100) per cent 'as effective as' the standard therapy, where R(LB) is the selected lower bound on the percentage effectiveness. A practical rationale is given for defining non-inferiority as a high fraction or percentage of the standard drug's efficacy, both in terms of statistical efficiency and medical relevance. For most typical 'at least as good as' applications (when R(LB)<R(True) < or =1), the ratio formatted test of H(0):R(True)< or =R(LB) is shown to be more efficient than Blackwelder's test of H(0): mu(st) - mu(et) > or = delta(BW), thereby requiring smaller sample sizes to detect the directionally based non-null alternatives contained in H(1): mu(et)/mu(st)>R(LB) or, equivalently, mu(st) - mu(et)<delta(BW). Further, when R(True)= 1.0, tests of Blackwelder's hypotheses, their ratio-based equivalents and conventional superiority can be evaluated for comparative efficiency. Testing H(0):R(True)< or =R(LB) with single-sided critical region of size alpha, versus H(1):R(True)>R(LB), is shown to be more efficient than excluding R(LB) from the lower limit of a 100(1-2alpha) per cent two-sided symmetric confidence interval centred by R. Relevant examples will be presented.

Non-inferiority trials: the ‘at least as good as’ criterion with dichotomous data

“…A similar phenomenon was pointed out in Reference [5] using continuous data. The ine ciency of Blackwelder's hypothesis test in (11), in similar circumstances, also pertains to the associated conÿdence interval procedure.…”

“…Brie y, the test requires that the e ect of a new therapy be no more than a ÿxed amount, 'delta' ( BW ), worse than the e ect of an active control. Recent extensions include 'adaptive' testing methods that allow the non-inferiority margin to vary as a function of the active control response [5][6][7]. In particular, the method proposed by Laster and Johnson [5] describes the statistical advantage of expressing the margin as a percentage of the control response, referred to as the 'at least as good as' criterion, based on the mean of a continuous response variable.…”

“…In this paper, as in Reference [5], Blackwelder's approach of testing et against st − BW will be recast as a test of et against a lower bound, a high percentage or fraction (R LB ) of st (R LB ¡1). In this format, the null and alternative hypotheses take the form…”

“…Phillips [6] conÿrms the form of this test statistic produced for an adaptive test for non-inferiority. The adaptive nature of Laster and Johnson's test for continuous data [5] as derived here for proportions, and Phillips' test [6], arises from the fact that the margin for non-inferiority varies as a function of the positive control response.…”

Bio‐creep in non‐inferiority clinical trials

“…Demanding 50% retention of the effect of the active comparator, with the hope of actually preserving this fraction of the efficacy, is one way of ensuring that is not the case. Additionally, doing so also allows one to infer efficacy of the new treatment as compared to placebo, even if the effect of the active control is markedly reduced from what was observed historically [6,7,8].…”