Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator

Pillonetto, Gianluigi; Chiuso, Alessandro

doi:10.1016/j.automatica.2015.05.012

Cited by 86 publications

(46 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parametric model, P, exhibits a poor performance because it describes only crude idealizations of the actual dynamics. The algorithms based on Cross Validation (CV) perform significantly worse in the first 60 seconds than those based on Marginal Likelihood (ML) optimisation; this is not unexpected as discussed in [53]. As expected, the nonparametric model, NP-ML, has worse generalization performance (the error is larger in the first few steps) but better adaptation capabilities with respect to model P. The models with the best performance are SP-ML and SPK-ML because they combine the benefit of the parametric approach, i.e.…”

Section: A Experimental Results Using Numerical Derivativesmentioning

confidence: 89%

Derivative-Free Online Learning of Inverse Dynamics Models

Romeres

Zorzi

Camoriano

et al. 2020

IEEE Trans. Contr. Syst. Technol.

Self Cite

View full text Add to dashboard Cite

This paper discusses online algorithms for inverse dynamics modelling in robotics. Several model classes including rigid body dynamics (RBD) models, data-driven models and semiparametric models (which are a combination of the previous two classes) are placed in a common framework. While model classes used in the literature typically exploit joint velocities and accelerations, which need to be approximated resorting to numerical differentiation schemes, in this paper a new "derivative-free" framework is proposed that does not require this preprocessing step. An extensive experimental study with real data from the right arm of the iCub robot is presented, comparing different model classes and estimation procedures, showing that the proposed "derivative-free" methods outperform existing methodologies.

show abstract

Section: A Experimental Results Using Numerical Derivativesmentioning

confidence: 89%

Derivative-Free Online Learning of Inverse Dynamics Models

Romeres

Zorzi

Camoriano

et al. 2020

IEEE Trans. Contr. Syst. Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…where det(·) is the determinant of a matrix. The EB (18) has the advantage that it is robust Pillonetto & Chiuso (2015), but not asymptotical optimal in the sense of MSE (Mu et al, 2017).…”

Section: Hyperparameter Estimationmentioning

confidence: 99%

“…We first use the method in Chen et al (2012); Pillonetto & Chiuso (2015) to generate 1000 30th order LTI systems. For each system, we truncate its impulse response at the order 50 and obtain a FIR model of order 50 accordingly, which is treated as the test system.…”

Section: Test Systemsmentioning

confidence: 99%

“…First, the kernel, through which the regularization is defined, provides a carrier for prior knowledge on the dynamic system to be identified. Second, the model complexity is tuned in a continuous manner through the hyperparameter, which is the parameter vector used to parameterize the kernel, Mu et al (2017); Pillonetto & Chiuso (2015). Third, extensive simulation results show that KRM can have better average accuracy and robustness than ML/PEM for the data that is short and/or has low signal-to-noise ratio, Chen et al (2012); Pillonetto et al (2014), and as a result, algorithms of KRM have been added to the System Identification Toolbox of MATLAB (Ljung et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On input design for regularized LTI system identification: Power-constrained input

Chen

2018

Automatica

View full text Add to dashboard Cite

Input design is an important issue for classical system identification methods but has not been investigated for the kernel-based regularization method (KRM) until very recently. In this paper, we consider in the time domain the input design problem of KRMs for LTI system identification. Different from the recent result, we adopt a Bayesian perspective and in particular make use of scalar measures (e.g., the A-optimality, D-optimality, and E-optimality) of the Bayesian mean square error matrix as the design criteria subject to power-constraint on the input. Instead to solve the optimization problem directly, we propose a two-step procedure. In the first step, by making suitable assumptions on the unknown input, we construct a quadratic map (transformation) of the input such that the transformed input design problems are convex, the number of optimization variables is independent of the number of input data, and their global minima can be found efficiently by applying well-developed convex optimization software packages. In the second step, we derive the expression of the optimal input based on the global minima found in the first step by solving the inverse image of the quadratic map. In addition, we derive analytic results for some special types of fixed kernels, which provide insights on the input design and also its dependence on the kernel structure.

show abstract

“…is the first order impulse response coefficient at lag τ i , E[·] denotes the expected value and 0 ≤ α, β ≤ ∞. The parameters c, α and β are called hyper-parameters and they are computed by maximizing the marginal likelihood of the observed output [11], [13], [8].…”

Section: Regularization For the First Order Volterra Kernelmentioning

confidence: 99%

Regularized nonparametric Volterra kernel estimation

et al. 2017

View full text Add to dashboard Cite

In this paper, the regularization approach introduced recently for nonparametric estimation of linear systems is extended to the estimation of nonlinear systems modelled as Volterra series. The kernels of order higher than one, representing higher dimensional impulse responses in the series, are considered to be realizations of multidimensional Gaussian processes. Based on this, prior information about the structure of the Volterra kernel is introduced via an appropriate penalization term in the least squares cost function. It is shown that the proposed method is able to deliver accurate estimates of the Volterra kernels even in the case of a small amount of data points.

show abstract

Tuning complexity in regularized kernel-based regression and linear system identification: The robustness of the marginal likelihood estimator

Cited by 86 publications

References 29 publications

Derivative-Free Online Learning of Inverse Dynamics Models

Derivative-Free Online Learning of Inverse Dynamics Models

On input design for regularized LTI system identification: Power-constrained input

Regularized nonparametric Volterra kernel estimation

Contact Info

Product

Resources

About