Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Abstract

We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.

The Constant Information Radar

“…This is important, as our predicted mutual information depends solely on the Markov tracking distributions. Therefore, as long as distributions can be formulated for a given scenario, the mutual information can be computed, or at least bounded [48], enabling the CIR to modulate revisit time.…”

“…More general tracking solutions can have complex distributions where the mutual information can be difficult to compute or estimate. In these cases, the CIR scheduling algorithm may still be used by applying reasonably tight bounds that can be formulated by modeling the filtering distributions as Gaussian mixture models GMMs [48] if the perturbative distributions are independent and additive.…”