Time Series Prediction via Aggregation: An Oracle Bound Including Numerical Cost

Abstract

We address the problem of forecasting a time series meeting the Causal Bernoulli Shift model, using a parametric set of predictors. The aggregation technique provides a predictor with well established and quite satisfying theoretical properties expressed by an oracle inequality for the prediction risk. The numerical computation of the aggregated predictor usually relies on a Markov chain Monte Carlo method whose convergence should be evaluated. In particular, it is crucial to bound the number of simulations needed to achieve a numerical precision of the same order as the prediction risk. In this direction we present a fairly general result which can be seen as an oracle inequality including the numerical cost of the predictor computation. The numerical cost appears by letting the oracle inequality depend on the number of simulations required in the Monte Carlo approximation. Some numerical experiments are then carried out to support our findings. Problem statement and main assumptionsReal stable autoregressive processes of a fixed order, referred to as AR(d) processes, are one of the simplest examples of CBS. They are defined as the stationary solution of(2.1)where the (ξ t ) t∈Z are i.i.d. real random variables with E[ξ t ] = 0 and E[ξ 2 t ] = 1.

Exponential inequalities for nonstationary Markov chains

“…Based on such inequalities, it is possible to prove generalization bounds for machine learning algorithms ; Steinwart and Christmann (2009); Shalizi and Kontorovich (2013); London et al (2014); Hang and Steinwart (2014); Sanchez-Perez (2015); Kuznetsov and Mohri (2015); McDonald et al (2017); Alquier and Guedj (2018). Model selection techniques in the spirit of Massart (2007) are studied in Meir (2000); Lerasle (2011); Alquier and Wintenberger (2012), and aggregation of estimators in Alquier et al (2013).…”

Deviation bound for non-causal machine learning

“…Model selection techniques, for instance, relies heavily on concentration inequality [Massart, 2007]. They have also been used for high dimensional procedures [Bickel et al, 2009, Alquier et al, 2020 or for studying different machine learning framework, such as time series prediction [Kuznetsov and Mohri, 2015], online machine learning [Sanchez-Perez, 2015] or classification problems [Freund et al, 2004]. Many concentration inequalities has been proposed for different framework and different hypothesis.…”