A Variable Projection Approach for Efficient Estimation of RBF-ARX Model

Abstract

The radial basis function network-based autoregressive with exogenous inputs (RBF-ARX) models have much more linear parameters than nonlinear parameters. Taking advantage of this special structure, a variable projection algorithm is proposed to estimate the model parameters more efficiently by eliminating the linear parameters through the orthogonal projection. The proposed method not only substantially reduces the dimension of parameter space of RBF-ARX model but also results in a better-conditioned problem. In this paper, both the full Jacobian matrix of Golub and Pereyra and the Kaufman's simplification are used to test the performance of the algorithm. An example of chaotic time series modeling is presented for the numerical comparison. It clearly demonstrates that the proposed approach is computationally more efficient than the previous structured nonlinear parameter optimization method and the conventional Levenberg-Marquardt algorithm without the parameters separated. Finally, the proposed method is also applied to a simulated nonlinear single-input single-output process, a time-varying nonlinear process and a real multiinput multioutput nonlinear industrial process to illustrate its usefulness.

Gradient Radial Basis Function Based Varying-Coefficient Autoregressive Model for Nonlinear and Nonstationary Time Series

“…It is now possible to optimize first with respect to the nonlinear parameters using (6), then the optimal parameters can be obtained by linear least-squares method [25], [26]. The Jacobian matrix of the vector of residuals can be computed analytically [27] or obtained by finite differences [28]. The nonlinear least squares algorithms, such as Gauss-Newton or Levenberg-Marquardt method, can be applied to solve (6).…”

Recursive parameter estimation methods and convergence analysis for a special class of nonlinear systems

“…Such methods in the works of Li et al and Hizir et al only study the parameter estimation for bilinear systems, while the state estimation is not taken into consideration. State estimation and filtering for nonlinear systems play an important role in control and signal processing . Thus, this paper studies the recursive state filtering and parameter estimation for the considered system.…”

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

“…The SNPOM was presented for estimating the RBF‐AR model by applying the parameter separation, the framework of the SNPOM is to update the nonlinear parameters with the Levenberg‐Marquardt algorithm and the linear parameters with the least squares algorithm . In order to reduce the dimension of the parameter space when estimating the RBF‐AR models, Gan et al proposed a VP method and the estimated model was exploited to predict a chaotic Mackey‐Glass time series . By using an analytical expression of the Jacobian matrix instead of finite differences, Chen et al developed the VP algorithm for the RBF‐AR models based on the modified Gram‐Schmidt method .…”

“…8 In order to reduce the dimension of the parameter space when estimating the RBF-AR models, Gan et al proposed a VP method and the estimated model was exploited to predict a chaotic Mackey-Glass time series. 9 By using an analytical expression of the Jacobian matrix instead of finite differences, Chen et al developed the VP algorithm for the RBF-AR models based on the modified Gram-Schmidt method. 6 The idea of the VP method was also employed to identify the GRBF-AR model which can handle a class of nonlinear nonstationary time series.…”