Noise Benefits in Combined Nonlinear Bayesian Estimators

Abstract

This paper investigates the benefits of intentionally adding noise to a Bayesian estimator, which comprises a linear combination of a number of individual Bayesian estimators that are perturbed by mutually independent noise sources and multiplied by a set of adjustable weighting coefficients. We prove that the Bayes risk for the mean square error (MSE) criterion is minimized when the same optimum weighting coefficients are assigned to the identical estimators in the combiner. This property leads to a simplified analysis of the noise benefit to the MSE of the combined Bayesian estimator even when the number of individual estimators tends to infinity. It is shown that, for a sufficiently large number of individual estimators, the MSE of the designed Bayesian estimator approaches a plateau for a wide range of added noise levels. This robust feature facilitates the incorporation of the added noise into the design of Bayesian estimators without tuning the noise level. For an easily implementable Bayesian estimator composed of quantizers, the benefit of the symmetric scale-family noise is demonstrated, and the optimal noise probability density function is approximated by solving a constrained nonlinear optimization problem. We further extend this potential Bayesian estimator to the nonlinear filter design. Finally, examples of the noise benefits in random parameter estimation and nonlinear filtering support the theoretical analyses.

Exploring $Bona~Fide$ Optimal Noise for Bayesian Parameter Estimation

“…Uhlich [42] also found that the optimal noise PDF, not limited to the noise type revealed in [28], [29], [32], [38], [39], has non-trivial complicated shapes. For minimizing the MSE of a combiner of identical estimators, we also found that solving the optimal noise PDF is a constrained nonlinear functional optimization problem, and approximate optimal PDFs of the optimal noise are also found to be complicated [46], [48].…”

“…This non-convex problem is in general intractable, because the term E x [E 2 η (g(x + η))] in (17) is a nonlinear functional of the PDF f η . Therefore, the minimization problem of the MSE min f η R NE usually employs the PDF approximation method [26], [42], [47], [48], [56] to obtain an approximate optimal solution form as…”

“…with the normalization coefficients λ k ≥ 0 satisfying the constraint K k=1 λ k = 1, and the Gaussian window function φ(u) = exp(−u 2 /2)/ √ 2π, means µ k , standard deviations σ k ≥ 0. The approximate PDFf o η will asymptotically converge to the existing optimal PDF f o η as the number K of the window function increases [26], [42], [47], [48], [56].…”

“…} is the covariance matrix of g [48], [57]. Then, using the optimal weights w 0 and w, the linear combination estimatorθ LC in the LMMSE sense can be rewritten aŝ…”

“…In this paper, we will theoretically provide the solutions to aforementioned crucial questions, and elucidate the possibility of exploiting the noise benefits in some easily implemented suboptimal estimators. We argue that the noise-enhanced Bayesian estimators proposed by [22]- [43], [45]- [49] in recent years can be mainly classified into four categories: (i) the noise-modified estimator established on a single sensor [32], (ii) a linear minimum MSE (LMMSE) estimator based on a single sensor, (iii) the noise-enhanced Bayesian estimator as the average of outputs of an ensemble of identical sensors [42] and (iv) the linear combination estimator executing the LMMSE transform on an array of identical or nonidentical sensors [48].…”