Learning from the past, predicting the statistics for the future, learning an evolving system

Abstract

Rough path theory is a mathematical toolbox providing for the deterministic modelling of interactions between highly oscillatory systems (rough paths). The theory is rich enough to capture and extend classical Itô stochastic calculus but has far wider significance. Fundamental to this approach is the realisation that the evolving state of the system is best described or measured over short time intervals by considering the realised effect of the system on certain controlled systems (the measurement instruments). The approach is crucially more efficient than an approach based on sampling. Even in the limit, the sequence of time series obtained through finer sampling need not provide a sufficient statistic that could predict the effects of a Brownian motion in an interaction ([13])! This divergence in the analysis arising from the choice of feature set is relevant in any context where the data streams are highly oscillatory on normal time scales.We bring the theory of rough paths to the study of non-parametric statistics on streamed data and particularly to the problem of regression where the input variable is a stream of information, and the dependent response is also (potentially) a path or a stream. We explain how a certain graded feature set of a stream, known in the rough path literature as the signature of the path, has a universality that allows one to characterise the functional relationship summarising the conditional distribution of the dependent response. At the same time this feature set allows explicit computational approaches through linear regression.Truncating the signature of a stream leads to an efficient local description for a rough path. The intrinsic non-linearity provides a provable efficiency gain for the signature over linear descriptions of the path

Market regime classification with signatures

“…In order to associate a signature to this set of points, some interpolation is required. Among several approaches proposed in the literature, we present here the piecewise linear and the rectilinear interpolations used by Levin, Lyons and Ni [11].…”

Framing RNN as a kernel method: A neural ODE approach

“…An essential ingredient towards our construction is the signature of a continuous-time process, which we briefly present here. We refer to Chevyrev and Kormilitzin (2016) for a gentle introduction and to Lyons et al (2007); Levin et al (2013) for details.…”

“…Although this definition is technical, the signature should simply be thought of as a feature map that embeds a bounded variation process into an infinite-dimensional tensor space. The signature has several good properties that make it a relevant tool for machine learning (e.g., Levin et al, 2013;Chevyrev and Kormilitzin, 2016;Fermanian, 2021). In particular, under certain assumptions, S(X) characterizes X up to translations and reparameterizations, and has good approximation properties.…”

“…Contributions. By taking advantage of the neural ODE paradigm for RNN, we show that RNN are, in the continuous-time limit, linear predictors over a specific space associated with the signature of the input sequence (Levin et al, 2013). The signature transform, first defined by Chen (1958) and central in rough path theory (Lyons et al, 2007;Friz and Victoir, 2010), summarizes sequential inputs by a graded feature set of their iterated integrals.…”

“…It was then extended to RNN in several articles, with a focus on handling irregularly sampled data (Rubanova et al, 2019;Kidger et al, 2020) and learning long-term dependencies (Chang et al, 2019). The signature transform has recently received the attention of the machine learning community (Levin et al, 2013;Kidger et al, 2019;Liao et al, 2019;Toth and Oberhauser, 2020;Fermanian, 2021) and, combined with deep neural networks, has achieved state-of-the-art performance for several applications (Yang et al, 2016(Yang et al, , 2017Perez Arribas, 2018;Wang et al, 2019;Morrill et al, 2020b). Király and Oberhauser (2019) use the signature transform to define kernels for sequential data and develop fast computational methods.…”