Survival analysis for AdVerse events with VarYing follow-up times (SAVVY)—estimation of adverse event risks

Abstract

Background The SAVVY project aims to improve the analyses of adverse events (AEs), whether prespecified or emerging, in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses, often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator are used, which ignore either censoring or CEs. In an empirical study including randomized clinical trials from several sponsor organizations, these potential sources of bias are investigated. The main purpose is to compare the estimators that are typically used to quantify AE risk within trial arms to the non-parametric Aalen-Johansen estimator as the gold-standard for estimating cumulative AE probabilities. A follow-up paper will consider consequences when comparing safety between treatment groups. Methods Estimators are compared with descriptive statistics, graphical displays, and a more formal assessment using a random effects meta-analysis. The influence of different factors on the size of deviations from the gold-standard is investigated in a meta-regression. Comparisons are conducted at the maximum follow-up time and at earlier evaluation times. CEs definition does not only include death before AE but also end of follow-up for AEs due to events related to the disease course or safety of the treatment. Results Ten sponsor organizations provided 17 clinical trials including 186 types of investigated AEs. The one minus Kaplan-Meier estimator was on average about 1.2-fold larger than the Aalen-Johansen estimator and the probability transform of the incidence density ignoring CEs was even 2-fold larger. The average bias using the incidence proportion was less than 5%. Assuming constant hazards using incidence densities was hardly an issue provided that CEs were accounted for. The meta-regression showed that the bias depended mainly on the amount of censoring and on the amount of CEs. Conclusions The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. We recommend using the Aalen-Johansen estimator with an appropriate definition of CEs whenever the risk for AEs is to be quantified and to change the guidelines accordingly.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) -- comparison of adverse event risks in randomized controlled trials

“…This is wellknown and has been shown either empirically, see Schuster et al [3] for a recent example in one single study, or analytically [4]. This paper builds on these and the one-sample results analyzed in the companion paper Stegherr et al [1] as follows: First, we consider the same six estimators of AE probabilities as in Stegherr et al [1] Second, we then extend the results on the bias of estimation of absolute AE probabilities in one sample when compared to the gold-standard Aalen-Johansen estimator to an assessment of how these same estimators perform when estimating RRs between two randomized treatment arms. With this, we answer a question raised in Unkel et al [5], namely which direction the bias goes for an estimator of a RR when based on biased one-sample estimators.…”

“…As one-sample estimators of the AE probability we use the incidence proportion, the probability transform incidence density ignoring and accounting for CE, one minus Kaplan-Meier, and the Aalen-Johansen estimator. A brief summary of one-sample estimators is given in the companion paper [1]. An even more detailed Statistical Analysis Plan is presented elsewhere [6].…”

“…For a time-to-event endpoint, the incidence density estimates the hazard function assuming that it is constant [6,1]. Their ratio therefore estimates a HR under the restrictive assumption that the hazard functions of both treatment arms are constant.…”

“…As a consequence, if we want to use a hazard-based analysis to quantify the effect of treatment on the event of interest and all CEs, in theory it is necessary to report all event-specific hazards, or rather the corresponding HRs. For this reason, we will not only report the performance of hazardbased RR estimators for the endpoint of interest, time to (first) AE, but also for the competing endpoints time to a CE of death and time to all CEs (see "Definition of CEs" in Stegherr et al [1] and section below). This is in perfect analogy to not only consider a (one minus Kaplan-Meier-like) simple probability transformation of the AE incidence density, but to also consider an (Aalen-Johansen-like) transformation accounting for CEs.…”

“…The motivation is that commonly employed methods to quantify absolute AE risk either do not account for varying follow-up times or for CEs and may be grouped into methods that either under-or overestimate cumulative AE probabilities in a time-to-first-event context. These concerns have been thoroughly investigated in a companion paper [1], considering only one trial arm in an opportunistic set of 186 types of AEs from seventeen RCTs from different disease indications. Here, we extend these considerations to comparing AE risk between arms, the challenge being that, say, the fact of overestimation in both arms of the same trial allows for both under-and overestimation of RRs when comparing arms.…”