Cramer-Rao Bound for Estimation After Model Selection and its Application to Sparse Vector Estimation

Abstract

In many practical parameter estimation problems, such as coefficient estimation of polynomial regression and direction-of-arrival (DOA) estimation, model selection is performed prior to estimation. In these cases, it is assumed that the true measurement model belongs to a set of candidate models. The data-based model selection step affects the subsequent estimation, which may result in a biased estimation. In particular, the oracle Cramér-Rao bound (CRB), which assumes knowledge of the model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we analyze the estimation performance of post-modelselection estimators, by using the mean-squared-selected-error (MSSE) criterion. We assume coherent estimators that force unselected parameters to zero, and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cramér-Rao-type bound on the MSSE and on the mean-squared-error (MSE) of any coherent and selective unbiased estimators. As an important special case, we illustrate the computation and applicability of the proposed selective CRB for sparse vector estimation, in which the selection of a model is equivalent to the recovery of the support. Finally, we demonstrate in numerical simulations that the proposed selective CRB is a valid lower bound on the performance of the post-model-selection maximum likelihood estimator for general linear model with different model selection criteria, and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the existing bounds: oracle CRB, averaged CRB, and the SMS-CRB from [1].

Low-Complexity Methods for Estimation After Parameter Selection

“…the considered cost function (see e.g. [3], [38], [50], [51]). The Lehmann unbiasedness for two-stage estimation after parameter selection w.r.t.…”

“…(38) According to (38), in the general case, even in this special case, all the parameters should be updated in order to obtain the second-best PSML. However, for the following scenarios we can update only the selected and the second best parameters, and set all the others to their ML estimators: A) For the independent populations model from Subsection II-B, the FIM, J x,y (θ), is a diagonal matrix.…”

“…However, for the following scenarios we can update only the selected and the second best parameters, and set all the others to their ML estimators: A) For the independent populations model from Subsection II-B, the FIM, J x,y (θ), is a diagonal matrix. Thus, in this scenario, (38) implies that only the estimators of θ m and θ m should be updated at each iteration via (38) and the other estimators of the parameters are equal to their ML value, θk (2B-PSML) (x, y) = θk (ML) (x, y), ∀k ∈ {1, . .…”