Dynamic error spectrum for estimation performance evaluation: a case study on interacting multiple model algorithm

Abstract

The commonly used root-mean-square error for estimation performance evaluation is easily dominated by large error terms. So many new alternative absolute metrics have been provided in X. R. Li's work. However, each of these metrics only reflects one narrow aspect of estimation performance, respectively. A comprehensive measure, error spectrum, was presented aggregating all these incomprehensive measures. However, when being applied to dynamic systems, this measure will have three dimensions over the total time span, which is not intuitive and difficult to be analysed. To overcome its drawbacks, a new metric, dynamic error spectrum (DES), is proposed in this study to extend the error spectrum measure to dynamic systems. Three forms under different application backgrounds are given, one of which is balanced taking into account both good and bad behaviour of an estimator and so can provide more impartial evaluation results. It can be applied to a variety of dynamic systems directly. Then the challenge in performance evaluation of the interacting multiple model (IMM) algorithm is considered, and the IMM algorithm is chosen as the testing case to illustrate the superiority of the DES metric. The simulation results validate its utility and effectiveness.

Radar Chart for Estimation Performance Evaluation

“…To solve the computation problem, Liu et al presented the Mellin transform to compute the ES analytically [6], then, we proposed two algorithms to calculate the ES based on the power means error and the Gaussian mixture model, respectively [7], [12], [13]. Furthermore, to solve the dynamic evaluation problem, Mao et al introduced a dynamic error spectrum (DES) to transform the ES into a single point at a time instant [8]. Obviously, the DES is an many-to-one mapping, which leads to the information loss problem.…”