Nonparametric sparse estimators for identification of large scale linear systems

Abstract

Identification of sparse high dimensional linear systems pose sever challenges to off-the-shelf techniques for system identification. This is particularly so when relatively small data sets, as compared to the number of inputs and outputs, have to be used. While input/output selection could be performed via standard selection techniques, computational complexity may however be a critical issue, being combinatorial in the number of inputs and outputs. Parametric estimation techniques which result in sparse models have nowadays become very popular and include, among others, the well known Lasso, LAR and their “grouped” versions Group Lasso and Group LAR. In this paper we introduce a new nonparametric technique which borrows ideas from a recently introduced Kernel estimator called “stable-spline” as well as from sparsity inducing priors which use l1 penalty. We compare the new method with a group LAR-type of algorithm applied to estimation of sparse Vector Autoregressive models and to standard PEM methods

VARX-L: Structured regularization for large vector autoregressions with exogenous variables

“…The Basic VARX-L (2.8), proposed by Chiuso and Pillonetto (2010), incorporates no structure and can be viewed as a special case of the Sparse Lag Group VARX-L in which α = 1, resulting in penalties of the form…”

Regularization and Bayesian learning in dynamical systems: Past, present and future

“…Consider a MIMO OE system (2) with transfer matrix 6 G(q); the matrix G(q) may have so zero entries (meaning that G ij (q) = 0), so that the i − th output does not depend on the j − th input. Detecting the zero entries of G(q) from data corresponds to performing variable selection, or equivalently finding the structure of a causality graph which describes the "interaction" between inputs and outputs; this problem has been framed in the context of sparsity in Materassi et al (2009); Chiuso and Pillonetto (2010b); Songsiri and Vandeberghe (2010); Sanandaji et al (2011), where the link with literature on sparse linear models (Tibshirani (1994); Efron et al (2004)) and compressive sensing (Donoho (2006)) has been exploited. Among the papers mentioned above, only Chiuso and Pillonetto (2010b) explicitly introduces regularization on the impulse responses to avoid overfitting when the number of lags T needs to be large.…”

Lasso Regularization Paths for NARMAX Models via Coordinate Descent

“…grouped Lasso [5] and elastic net [6]) for building models of dynamic systems and time series have received considerable attention in the last years. They have been used for the identification of nonparametric [7], [8], polynomial [9] and posynomial [10] dynamic models. In [11], [12] its application to autoregressive time series was studied, with the estimator properties derived.…”