Compressive System Identification in the Linear Time-Invariant framework

Abstract

Abstract-Selection of an efficient model parametrization (model order, delay, etc.) has crucial importance in parametric system identification. It navigates a trade-off between representation capabilities of the model (structural bias) and effects of over-parametrization (variance increase of the estimates). There exists many approaches to this widely studied problem in terms of statistical regularization methods and information criteria. In this paper, an alternative ℓ1 regularization scheme is proposed for estimation of sparse linear-regression models based on recent results in compressive sensing. It is shown that the proposed scheme provides consistent estimation of sparse models in terms of the so-called oracle property, it is computationally attractive for large-scale over-parameterized models and it is applicable in case of small data sets, i.e., underdetermined estimation problems. The performance of the approach w.r.t. other regularization schemes is demonstrated in an extensive Monte Carlo study.

A linear programming approach to sparse linear regression with quantized data

“…In machine learning and system identification, sparsity is desirable to reduce as much as possible the complexity of the estimated models. In the literature, sparsity is exploited for the identification of linear systems, see, e.g., [3], [4], [5]; non-linear functions in [6]; polynomial models in [7]; time-varying systems in [8], [9].…”

On robustness of &#x2113;<inf>1</inf>-regularization methods for spectral estimation

“…In this subsection we will bound the transportation dis-tanceT (x est , x true ) for any two vectors x true and x est satisfying (11). The proof idea is to show that there exists a vectorx ∈ C K such that…”

“…Lemma 4 (Main Lemma): Let x true , x est be vectors satisfying (11). Then there existsx ∈ C K×1 with x 1 = x est 1 and where the support ofx is a subset of Λ, such that…”

“…, θ K }. The system (2) is underdetermined and hence has infinitely many solutions; and to single out a unique solution, sparse methods use 1 regularization as a convex surrogate for the cardinality (see, e.g., the review articles [6], [7]; and [8], [9], [10], [11] for applications to signals and systems). This sparse recovery problem has been intensely studied during the last decade for various classes of underdetermined systems and significant advances have been made in understanding when these approaches recover a sparse signal.…”

A CS Recovery Algorithm for Model and Time Delay Identification of MISO-FIR Systems

“…The sparse recovery problem can be solved by the l 1 -norm convex relaxation. However, the l 1 regularization schemes are always complex, requiring heavy computational burdens [20][21][22]. The greedy methods have speed advantages and are easily implemented.…”