Hidden Relationships: Bayesian Estimation With Partial Knowledge

Abstract

Abstract-We address the problem of Bayesian estimation where the statistical relation between the signal and measurements is only partially known. We propose modeling partial Bayesian knowledge by using an auxiliary random vector called instrument. The statistical relations between the instrument and the signal and between the instrument and the measurements, are known. However, the joint probability function of the signal and measurements is unknown. Two types of statistical relations are considered, corresponding to second-order moment and complete distribution function knowledge. We propose two approaches for estimation in partial knowledge scenarios. The first is based on replacing the orthogonality principle by an oblique counterpart and is proven to coincide with the method of instrumental variables from statistics, although developed in a different context. The second is based on a worst-case design strategy and is shown to be advantageous in many aspects. We provide a thorough analysis showing in which situations each of the methods is preferable and propose a nonparametric method for approximating the estimators from a set of examples. Finally, we demonstrate our approach in the context of enhancement of facial images that have undergone unknown degradation and image zooming.Index Terms-Bayesian estimation, instrumental variables, minimax regret, nonparametric regression, partial knowledge.

Partially Linear Estimation With Application to Sparse Signal Recovery From Measurement Pairs

“…We consider an estimator minimax-optimal if its worst-case MSE over the set of all feasible distributions is minimal. For example, the LMMSE estimator attains the minimal possible worst-case MSE over the set of distributions with given first-and second-order moments [8]. In the next theorem, we show that the PLMMSE method is optimal in the sense that its worst-case MSE over the set of all distributions complying with the knowledge appearing in Fig.…”

“…However, the fact that the PLMMSE estimator is merely determined by E[X|Z], Cov(X, Y ) and F Y Z (y, z), does not yet imply that it is optimal among all methods that rely solely on these quantities. The question of optimality of an estimator with respect to partial knowledge regarding the joint distribution of the signal and measurements was recently addressed in [7]. One of the notions of optimality considered there, which we adopt here as well, follows from a worst-case perspective.…”

“…We consider an estimator minimax-optimal if its worst-case MSE over the set of all feasible distributions is minimal. For example, it was shown in [7] that the LMMSE estimator XL Y attains the minimal possible worst-case MSE over the set of distributions F XY (x, y) with given first-and second-order moments.…”

Resource Allocation with Incomplete Information for QoE-Driven Multimedia Communications

“…While Bayesian approach is able to provide an efficient means (e.g., relatively low complexity) for the testoptimization problem, however, it has some non-negligible deficiencies. Most notably, the dynamic optimization problem largely depends on the prior distribution of the unknown parameters [14], which may be still unavailable for practical multimedia communications. On the other hand, stochastic approximation estimates the unknown utility function based on large historical data [15]- [17], and [18] shows that it can achieve near-optimal solution in the long run.…”