Estimator selection: End-performance metric aspects

Abstract

Abstract-Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs based on usual pointwise functional distances of the estimated system from the true one. The separation of the system estimator from the experiment design is done within this new framework by choosing and fixing the estimation method to either a maximum likelihood (ML) approach or a Bayesian estimator such as the minimum mean square error (MMSE). Since the MMSE estimator delivers a system estimate with lower mean square error (MSE) than the ML estimator for finite-length experiments, it is usually considered the best choice in practice in signal processing and control applications. Within the application-oriented framework a related meaningful question is: Are there endperformance metrics for which the ML estimator outperforms the MMSE when the experiment is finite-length? In this paper, we affirmatively answer this question based on a simple linear Gaussian regression example. Recently, a new framework for performing experiment design has been introduced. This framework is termed application-oriented experiment design and it has been outlined in [8]. Specific investigations related to communication systems were performed in [11], [12]. Denoting the endperformance metric by J and assuming that J depends on the true and the estimated models, the performance is considered to be acceptable if J ≤ 1/γ for some parameter γ, which we call accuracy. This motivates the introduction of a set of admissible models E adm = {G : J ≤ 1/γ}, where G denotes the model to be inferred. With these definitions, the leastcostly experiment is formulated as follows: I. INTRODUCTION