Weakly convergent nonparametric forecasting of stationary time series

Abstract

The conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past. Several schemes are known for constructing pointwise consistent estimates, but they all demand prohibitive amounts of input data. In this paper we consider real-valued time series and construct conditional distribution estimates that make much more efficient use of the input data. The estimates are consistent in a weak sense, and the question whether they are pointwise consistent is still open. For finite-alphabet processes one may rely on a universal data compression scheme like the Lempel-Ziv algorithm to construct conditional probability mass function estimates that are consistent in expected information divergence. Consistency in this strong sense cannot be attained in a universal sense for all stationary processes with values in an infinite alphabet, but weak consistency can. Some applications of the estimates to on-line forecasting, regression and classification are discussed.

Universal codes as a basis for time series testing

“…From definitions (23) and (27) we can see that the Laplace predictor ascribes the following probabilities:…”

“…The theory of universal coding, which is a part of Information Theory, also has been efficiently applied to many fields since its discovery in [10,18]. Thus, application of results of universal coding, initiated in [29], created a new approach to prediction [15,23,24].…”

Universal codes as a basis for nonparametric testing of serial independence for time series

“…More precisely, for any predictor there exists a stationary and ergodic sourcep, 9 such that lim sup t→∞ ,p (x 1 · · · x t ) const > 0 with probability 1; Ryabko (1988). (See also Algoet, 1999;Morvai et al, 1997;Nobel, 2003, where this result is generalized and 11 the history of its discovery is described. In particular, they found out that such a result was described by Bailey, 1976 in his unpublished thesis.)…”

Forward estimation for ergodic time series

“…Bailey [5], Ornstein [20], and even for a more general case Morvai, Yakowitz, Györfi [19], Algoet [1], Györfi and Lugosi [9]) and there are schemes that solve Probkem 5 (e.g. Bailey [5], Ornstein [20] and for even more general case Morvai, Yakowitz, Györfi [19], Algoet [1], Morvai, Yakowitz and Algoet [18]). In this paper we propose a reasonable, very simple algorithm that simultanously solves Problem 3, 4 and 5.…”