Optimal subsampling for parametric accelerated failure time models with massive survival data

Abstract

With increasing availability of massive survival data, researchers need valid statistical inferences for survival modeling whose computation is not limited by computer memories. Existing works focus on relative risk models using the online updating and divide-and-conquer strategies. The subsampling strategy has not been available due to challenges in developing the asymptotic properties of the estimator under semiparametric models with censored data. This article tackles optimal subsampling algorithms to fast approximate the maximum likelihood estimator for parametric accelerate failure time models with massive survival data. We derive the asymptotic distributions of the subsampling estimator and the optimal sampling probabilities that minimize the asymptotic mean squared error of the estimator. A feasible two-step algorithm is proposed where the optimal sampling probabilities in the second step are estimated based on a pilot sample in the first step. The asymptotic properties of the two-step estimator are established. The performance of the estimator is validated in a simulation study. A real data analysis illustrates the usefulness of the methods.

Subsampling approach for least squares fitting of semi-parametric accelerated failure time models to massive survival data

“…This approach has been extended to a variety of statistical models such as generalized linear models (Ai et al, 2021) and quantile regression models (Wang and Ma, 2021). In the field of survival analysis, this approach has been developed for Cox models (Zhang et al, 2022), Cox models with rare events (Keret and Gorfine, 2022), additive hazard rate models (Zuo et al, 2021), and parametric AFT models (Yang et al, 2022). To the best of our knowledge, however, no prior work has explored its application to semi-parametric AFT models for massive survival data.…”

“…In the presence of censoring, the key challenge is to derive the optimal subsampling probability (SSP) for censored observations. The SSP of an observation is proportional to its contribution to the estimating functions in standard approaches (Zhang et al, 2022;Yang et al, 2022). For the rank-based method, it is tempting to assign a zero SSP to censored observations since they do not contribute to the estimating functions for the regression coefficients.…”

“…For the least squares method, however, the contribution of a censored observation to the estimating equations has an explicit form (Tsiatis, 1990). Conceptually, the optimal SSPs are expected to behave similarly to those in a parametric AFT model (Yang et al, 2022). The extra challenge comes from the evaluation of these contributions.…”

Goodness‐of‐fit test for a parametric mixture cure model with partly interval‐censored data

“…Popular parametric distributions for survival times include the Weibull distribution, the Gompertz distribution, the lognormal distribution, the loglogistic distribution, among many others 1,28‐31 . Recently, a Weibull model is applied 32 to a very large lymphoma data set ($$$ \approx 1{0}^6 $$$ records) using the online updating and divide‐and‐conquer strategies. A parametric model provides a feasible solution to analyze such massive survival data which are increasingly available with the rapid development of surveillance and storage technologies.…”