Learning Linear Dynamical Systems with Semi-Parametric Least Squares

Abstract

We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance introduced by long-term dependencies. We then demonstrate that prefiltered least squares yields, to our knowledge, the first algorithm that provably estimates the parameters of partially-observed linear systems that attains rates which do not not incur a worst-case dependence on the rate at which these dependencies decay. The algorithm is provably consistent even for systems which satisfy the weaker marginal stability condition obeyed by many classical models based on Newtonian mechanics. In this context, our semi-parametric framework yields guarantees for both stochastic and worst-case noise.

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems

“…In the partially observable linear systems, similar to the trend in fully observable counterparts, most of the prior works focus on the system identification aspects [Ljung, 1999, Chen et al, 1992, Juang et al, 1993, Phan et al, 1994, Lee and Zhang, 2019, Oymak and Ozay, 2018, Sarkar et al, 2019, Simchowitz et al, 2019, Lee and Lamperski, 2019, Umenberger et al, 2019, Tsiamis and Pappas, 2020. A body of work aimed to extend the problem of estimation and prediction to online convex optimization where a set of strong theoretical guarantees on cumulative prediction errors are provided [Abbasi-Yadkori et al, 2014, Hazan et al, 2017, Arora et al, 2018, Hazan et al, 2018, Lee and Zhang, 2019, Ghai et al, 2020 Building upon the system identification algorithms, Lale et al [2020a] provides Õ(T 2/3 ) regret upper bound in system with stochastic noise using OFU.…”

“…In the setting where the observations of the system's state evolution are partial and noisy, learning the dynamics of linear systems brings a series of challenges due to lack of direct access to the underlying events. For partially observable systems, a variety of methods have been proposed to learn the open-loop system dynamics via exciting the system with random and uncorrelated noise for long enough that the regression methods provide efficient estimations of the model parameters [Oymak and Ozay, 2018, Sarkar et al, 2019, Simchowitz et al, 2019.…”

Linear Systems can be Hard to Learn

“…With the advances in high-dimensional statistics [6], there has been a recent shift from asymptotic analysis with infinite data to statistical analysis of system identification with finite samples. Over the past two years there have been significant advances in understanding finite sample system identification for both fully-observed systems [7][8][9][10][11][12][13][14] as well as partially-observed systems [15][16][17][18][19][20][21][22][23][24]. A tutorial can be found in [25].…”

Task-Optimal Exploration in Linear Dynamical Systems

“…In control, results have focused on finite time regret bounds for the LQR problem with unknown dynamics Dean et al, 2017Dean et al, , 2018Mania et al, 2019;Dean et al, 2019;Cohen et al, 2019), with Simchowitz & Foster (2020) ultimately settling the minimax optimal regret in terms of dimension and time horizon; others have considered regret in online adversarial settings (Agarwal et al, 2019;. Recent results in system identification have focused on obtaining finite time high probability bounds on the estimation error of the system's parameters when observing the evolution over time (Tu et al, 2017;Faradonbeh et al, 2018;Hazan et al, 2018;Hardt et al, 2018;Simchowitz et al, 2018;Sarkar & Rakhlin, 2018;Oymak & Ozay, 2019;Simchowitz et al, 2019;Tsiamis & Pappas, 2019). Existing results rely on excitation from random noise to guarantee learning and do not consider the problem of learning with arbitrary sequences of inputs or optimally choosing inputs for excitation.…”