An online updating approach for testing the proportional hazards assumption with streams of survival data

Abstract

The Cox model—which remains the first choice for analyzing time‐to‐event data, even for large data sets—relies on the proportional hazards (PH) assumption. When survival data arrive sequentially in chunks, a fast and minimally storage intensive approach to test the PH assumption is desirable. We propose an online updating approach that updates the standard test statistic as each new block of data becomes available and greatly lightens the computational burden. Under the null hypothesis of PH, the proposed statistic is shown to have the same asymptotic distribution as the standard version computed on an entire data stream with the data blocks pooled into one data set. In simulation studies, the test and its variant based on most recent data blocks maintain their sizes when the PH assumption holds and have substantial power to detect different violations of the PH assumption. We also show in simulation that our approach can be used successfully with “big data” that exceed a single computer's computational resources. The approach is illustrated with the survival analysis of patients with lymphoma cancer from the Surveillance, Epidemiology, and End Results Program. The proposed test promptly identified deviation from the PH assumption, which was not captured by the test based on the entire data.

Sampling‐based estimation for massive survival data with additive hazards model

“…Hence, we recommend the OSP when applying our method in practical applications. In conclusion, it is desirable to choose our subsampling approach over the methods of Kawaguchi et al 12 or Xue et al 14 when we have limited computing resources at hand.…”

“…conclusion, it is desirable to choose our subsampling approach over the methods of Kawaguchi et al 12 or Xue et al 14 when we have limited computing resources at hand. Of note, the UNIF approach is different from bootstrap.…”

“…As a competitive alternative to the Cox proportional hazards (PH) model, the additive hazards (AH) model 15,16 has several advantages: examining additive associations vs multiplicative associations, not assuming PH, and avoiding issues with the interpretation of the hazard ratio. These advantages may also scale well to the massive data case, while Xue et al 14 demonstrated the complexity of examining PH with massive data. To the best of our knowledge, subsampling procedures have not been developed for censored survival data.…”

“…Wang et al 13 proposed an efficient “divide and conquer” algorithm to fit sparse Cox regression with massive datasets. Xue et al 14 proposed an online updating approach for testing the proportional hazards assumption with streams of survival data.…”

Divide-and-Conquer Information-Based Optimal Subdata Selection Algorithm

“…The divide-and-conquer method has attracted many researchers in machine learning and statistics, leading to key advancements in Lin and Xie (2011); Jordan (2012); Chen and Xie (2014); Shang and Cheng (2017); Battey et al (2018), among others. In the streaming setting where data blocks are accessible sequentially for only one time, the online updating method has been developed (Schifano et al, 2016;Xue et al, 2018). In order to produce an estimator that preserves the same convergence rate as the full data estimator, a key requirement for the divide-and-conquer method is that the number of partitions cannot be too large (Shang and Cheng, 2017).…”