Multivariate online regression analysis with heterogeneous streaming data

Abstract

New data collection and storage technologies have given rise to a new field of streaming data analytics, called real-time statistical methodology for online data analyses. Most existing online learning methods are based on homogeneity assumptions, which require the samples in a sequence to be independent and identically distributed. However, inter-data batch correlation and dynamically evolving batch-specific effects are among the key defining features of real-world streaming data such as electronic health records and mobile health data. This article is built under a state-space mixed model framework in which the observed data stream is driven by a latent state process that follows a Markov process. In this setting, online maximum likelihood estimation is made challenging by high-dimensional integrals and complex covariance structures. In this article, we develop a real-time Kalman-filter-based regression analysis method that updates both point estimates and their standard errors for fixed population average effects while adjusting for dynamic hidden effects. Both theoretical justification and numerical experiments demonstrate that our proposed online method has statistical properties similar to those of its offline counterpart and enjoys great computational efficiency. We also apply this method to analyze an electronic health record dataset.

Adaptive Regression Analysis of Heterogeneous Data Streams via Models with Dynamic Effects

“…However, the underlying model structures for different data batches are the same, that is, a common generalized linear model. A similar strategy has been used in Luo and Song [18], although they only focus on linear models and impose stricter restrictions on the patterns of how coefficients change.…”

“…The first one is that loading the whole dataset O * b is infeasible since the previous data O * b−1 is not available any more in an online setting; that is, we only have access to the current data batch O b and a set of historical summary statistics, denoted by H b−1 . Another one is that at each accumulation point b, it is possible that we have θ b = θ b−1 , meaning that θ b − θ b−1 has at least one nonzero component [18]. Due to the potential dynamic coefficients of the current data batch O b , using the undesirable historical information H b−1 directly might lead to incorrectly estimated regression coefficients for those we are interested in at the latest moment.…”

“…Nevertheless, most of the existing methods for analyzing streaming datasets are designed by assuming that all sequentially available data batches are sampled from the same pre-specified model structure with common unknown parameters. Since this widely adopted assumption for streaming data might be violated in many real-world applications, it is important to consider heterogeneity or data drift, which means that underlying model specifications are likely to be changed over time [18]. Directly applying the online methods that ignore this issue may invalidate the model accuracy for the current data batch due to the potentially incorrect estimates of the parameters of interest.…”

Renewable quantile regression for streaming datasets

“…Wu et al (2021) proposed an online updating method of survival analysis under the Cox proportional hazards model. Luo and Song (2021) studied a multivariate online regression analysis with heterogeneous streaming data. Lin et al (2021) studied a homogenization strategy for heterogeneous streaming data.…”

Turning the information-sharing dial: efficient inference from different data sources

“…Similarly, transfer learning for high-dimensional regression in generalized linear models aims to improve predictions and generally does not provide valid inference (Li et al, 2020(Li et al, , 2022Tian and Feng, 2022). Another viewpoint casts this problem in an online inference framework (Schifano et al, 2016;Toulis and Airoldi, 2017;Luo and Song, 2021) that assumes the true value of the parameter of interest in two sequentially considered datasets is the same.…”