A weighted least-squares method for parameter estimation in structured models

Abstract

Abstract-Parameter estimation in structured models is generally considered a difficult problem. For example, the prediction error method (PEM) typically gives a non-convex optimization problem, while it is difficult to incorporate structural information in subspace identification. In this contribution, we revisit the idea of iteratively using the weighted least-squares method to cope with the problem of non-convex optimization. The method is, essentially, a three-step method. First, a high order least-squares estimate is computed. Next, this model is reduced to a structured estimate using the least-squares method. Finally, the structured estimate is re-estimated, using weighted least-squares, with weights obtained from the first structured estimate. This methodology has a long history, and has been applied to a range of signal processing problems. In particular, it forms the basis of iterative quadratic maximum likelihood (IQML) and the Steiglitz-McBride method. Our contributions are as follows. Firstly, for output-error models, we provide statistically optimal weights. We conjecture that the method is asymptotically efficient under mild assumptions and support this claim by simulations. Secondly, we point to a wide range of structured estimation problems where this technique can be applied. Finally, we relate this type of technique to classical prediction error and subspace methods by showing that it can be interpreted as a link between the two, sharing favorable properties with both domains.

A Least Squares Method for Identification of Feedback Cascade Systems**This work was supported by the European Research Council under the advanced grant LEARN, contract 267381 and by the Swedish Research Council under contract 621-2009-4017.

“…To deal with that limitation, we propose a method that uses the approach in Galrinho et al (2014), and transforms the problem into one where the indirect PEM algorithm can be applied. Analogously to the acyclic case, measuring all outputs is not a necessary condition to obtain all the transfer functions.…”

“…The purpose of this section is to propose an intermediate step, based on the method presented in Galrinho et al (2014), which reduces the problem to a setting that allows the application of the indirect PEM algorithm as in the previous sections.…”

“…In Section 4, we point out that this method can be applied even if not all the transfer function outputs in a network are measured. Then, we consider a feedback cascade structure in Section 5, for which indirect PEM does not avoid non-convexity, and propose an intermediate step using the method in Galrinho et al (2014). A numerical simulation is presented in Section 6, followed by a discussion in Section 7.…”

Combined Estimation and Optimal Control of Batch Membrane Processes

“…Compared to the standard least-squares method, the WLS method considers that different measured outputs can contribute differently to the estimation of unknown parameters based on their variability and statistical significance [32]. For the on-line parameter estimation, it must be, however, considered that with each new measurement, a new NLP problem must be solved, and moreover, if a large set of experimental points is available, numerical problems can occur due to the gradually increasing complexity of the optimization problem.…”

On estimating initial conditions in unstructured models

“…Some methods estimate an unstructured high-order model as an intermediate step to obtain the model of interest. Examples are the methods in [8] and [9], and also some subspace identification methods, such as SSARX [10].…”