A new Ultra Least Squares (ULS) criterion is introduced for system identification. Unlike the standard least squares criterion which is based on the Euclidean norm of the residuals, the new ULS criterion is derived from the Sobolev space norm. The new criterion measures not only the discrepancy between the observed signals and the model prediction but also the discrepancy between the associated weak derivatives of the observed and the model signals. The new ULS criterion possesses a clear physical interpretation and is easy to implement. Based on this, a new Ultra Orthogonal Forward Regression (UOFR) algorithm is introduced for nonlinear system identification, which includes converting a least squares regression problem into the associated ultra least squares problem and solving the ultra least squares problem using the orthogonal forward regression method. Numerical simulations show that the new UOFR algorithm can significantly improve the performance of the classic OFR algorithm.Key words: orthogonal forward regression, system identification, ultra least squares, ultra orthogonal forward regression, ultra orthogonal least squares.
IntroductionSystem identification plays a more and more important role in revealing the unknown mechanisms and rules underlying complex phenomena (Schmidt & Lipson, 2009). System identification includes the detection of the model structure and estimation of the associated parameters. A system identification problem can often be thought of as an optimization problem where the optimal model is searched from a large predefined candidate model dictionary given a criterion. The criterion is 2 used to evaluate the performance of each model by measuring the discrepancy between the observed data and the model predictions. The candidate model dictionary is often chosen to be large enough to include the unknown correct model. Hence an exhaustive search algorithm is often infeasible in these kinds of applications because of the large solution space. Even an evolutionary algorithm which can greatly reduce the search process can still be very computationally intensive.Hence an algorithm which can efficiently find the optimal solution is desired. However, a fast algorithm often dictates an optimal substructure; otherwise the search may converge to a suboptimal solution. Many efforts have been made to improve the search process under a certain specific loss function or performance index, for example, the simulated annealing algorithm, particle swarm optimisation, and so on. In this paper, a different and new methodology will be introduced.Instead of improving the search method, a new and effective criterion will be introduced to describe the objective of the regression more accurately. Under the new criterion, the solution space has a better structure and a fast algorithm is more likely to find the optimal solution.System identification aims to identify a model from observed data based on a criterion. A good criterion results in not only better parameter estimation but also a good search pa...