Prediction of dynamical time series using kernel based regression and smooth splines

Abstract

Prediction of dynamical time series with additive noise using support vector machines or kernel based regression is consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the present coordinates. However, the present coordinates themselves are noisy, and therefore, these methods are not necessarily effective at removing noise. In this article, we consider denoising and prediction as separate problems for flows, as opposed to discrete time dynamical systems, and show that the use of smooth splines is more effective at removing noise. Combination of smooth splines and kernel based regression yields predictors that are more accurate on benchmarks typically by a factor of 2 or more. We prove that kernel based regression in combination with smooth splines converges to the exact predictor for time series extracted from any compact invariant set of any sufficiently smooth flow. As a consequence of convergence, one can find examples where the combination of kernel based regression with smooth splines is superior by even a factor of 100. The predictors that we analyze and compute operate on delay coordinate data and not the full state vector, which is typically not observable.

Alternative stable states, nonlinear behavior, and predictability of microbiome dynamics

“…The optimal number of embedding dimensions 29,39 ( E ) was obtained by finding E giving the smallest root-mean-square error (RMSE) in pre-run forecasting with simplex projection 20 or S-map 19 as detailed below. Taking into account a previous study examining embedding dimensions 62 , optimal E was explored within the range from 1 to 20. Prior to the embedding, all the variables were z -standardized (i.e., zero-mean and unit-variance) across the time-series of each ASV in each replicate community.…”

Alternative stable states, nonlinear behavior, and predictability of microbiome dynamics

“…The optimal number of embedding dimensions [ 29 , 38 ] ( E ) was obtained by finding E giving the smallest root-mean-square error (RMSE) in pre-run forecasting with simplex projection [ 20 ] or S-map [ 19 ] as detailed below. Taking into account a previous study examining embedding dimensions [ 64 ], optimal E was explored within the range from 1 to 20. Prior to the embedding, all the variables were z -standardized (i.e., zero-mean and unit-variance) across the time-series of each ASV in each replicate community.…”

The Bootstrap for Dynamical Systems

“…This is carried out in a variety of fields and has wide ranging applications; for earlier surveys, see [4], [11], [35], [36]; for a recent review with numerous references, see [53]. There is also an increasing trend to study the estimation and prediction in dynamical systems theoretically, and several recent works in this vein include [29], [52], [54], [55], [58], [73]. They present the consistency and/or the rate of convergence in various point estimation or prediction settings.…”