Interpretation of active-control randomised trials: the case for a new analytical perspective involving averted events

Abstract

Active-control trials, where an experimental treatment is compared with an established treatment, are performed when the inclusion of a placebo control group is deemed to be unethical. For time-to-event outcomes, the primary estimand is usually the rate ratio, or the closely-related hazard ratio, comparing the experimental group with the control group. In this article we describe major problems in the interpretation of this estimand, using examples from COVID-19 vaccine and HIV pre-exposure prophylaxis trials. In particular, when the control treatment is highly effective, the rate ratio may indicate that the experimental treatment is clearly statistically inferior even when it is worthwhile from a public health perspective. We argue that it is crucially important to consider averted events as well as observed events in the interpretation of active-control trials. An alternative metric that incorporates this information, the averted events ratio, is proposed and exemplified. Its interpretation is simple and conceptually appealing, namely the proportion of events that would be averted by using the experimental treatment rather than the control treatment. The averted events ratio cannot be directly estimated from the active-control trial, and requires an additional assumption about either: (a) the incidence that would have been observed in a hypothetical placebo arm (the counterfactual incidence) or (b) the efficacy of the control treatment (relative to no treatment) that pertained in the active-control trial. Although estimation of these parameters is not straightforward, this must be attempted in order to draw rational inferences. To date, this method has been applied only within HIV prevention research, but has wider applicability to treatment trials and other disease areas.