Impact of limited sample size and follow-up on single event survival extrapolation for health technology assessment: a simulation study

Abstract

Introduction Extrapolation of time-to-event data from clinical trials is commonly used in decision models for health technology assessment (HTA). The objective of this study was to assess performance of standard parametric survival analysis techniques for extrapolation of time-to-event data for a single event from clinical trials with limited data due to small samples or short follow-up. Methods Simulated populations with 50,000 individuals were generated with an exponential hazard rate for the event of interest. A scenario consisted of 5000 repetitions with six sample size groups (30–500 patients) artificially censored after every 10% of events observed. Goodness-of-fit statistics (AIC, BIC) were used to determine the best-fitting among standard parametric distributions (exponential, Weibull, log-normal, log-logistic, generalized gamma, Gompertz). Median survival, one-year survival probability, time horizon (1% survival time, or 99th percentile of survival distribution) and restricted mean survival time (RMST) were compared to population values to assess coverage and error (e.g., mean absolute percentage error). Results The true exponential distribution was correctly identified using goodness-of-fit according to BIC more frequently compared to AIC (average 92% vs 68%). Under-coverage and large errors were observed for all outcomes when distributions were specified by AIC and for time horizon and RMST with BIC. Error in point estimates were found to be strongly associated with sample size and completeness of follow-up. Small samples produced larger average error, even with complete follow-up, than large samples with short follow-up. Correctly specifying the event distribution reduced magnitude of error in larger samples but not in smaller samples. Conclusions Limited clinical data from small samples, or short follow-up of large samples, produce large error in estimates relevant to HTA regardless of whether the correct distribution is specified. The associated uncertainty in estimated parameters may not capture the true population values. Decision models that base lifetime time horizon on the model’s extrapolated output are not likely to reliably estimate mean survival or its uncertainty. For data with an exponential event distribution, BIC more reliably identified the true distribution than AIC. These findings have important implications for health decision modelling and HTA of novel therapies seeking approval with limited evidence.

Sedentary time and the risk of metabolic syndrome: A systematic review and dose–response meta‐analysis

“…Second, in the dose–response analysis of two prospective cohort studies, although a marginally significant linear association was observed, the exposure levels of Greer's study showed a steady increase in effect sizes, but Honda's study showed a nonlinear association. It is possible that the nonlinear dose–response association of Honda's study was linearized due to the relatively larger sample size of Greer's study 68–70 . The low number of studies included and insufficient data extracted may be other important reasons.…”

“…It is possible that the nonlinear dose-response association of Honda's study was linearized due to the relatively larger sample size of Greer's study. [68][69][70] The low number of studies included and insufficient data extracted may be other important reasons. Therefore, further cohort studies with large sample sizes are urgently needed to shed light on the linearity or nonlinearity of the association between sedentary time and MetS risk.…”