Log Ratio of Entropy Powers

Mutual Information, the Linear Prediction Model, and CELP Voice Codecs

“…These results are from [8] and extend the data processing inequality by providing a new characterization of the information loss between stages in terms of the entropy powers of the two stages. Since differential entropies are difficult to calculate, it would be particularly useful if we could obtain expressions for the entropy power at two stages and then use Equations (13) and (17) to find the difference in differential entropy and mutual information between these stages.…”

“…An interesting quantity introduced and analyzed by Gibson in a series of papers is the log ratio of entropy powers [5,8,9]. Specifically, the log ratio of entropy powers is related to the difference in mutual information, and further, in many cases, the entropy powers can be replaced with the minimum mean squared prediction error (MMSPE) in the ratio.…”

“…In the context of Equation 9, the notation Q X|Y n denotes the minimum variance when reconstructing an approximation to X given the sequence at the output of stage n in the chain [8].…”

“…We can use the definition of the entropy power in Equation (6) to express the logarithm of the ratio of two entropy powers in terms of their respective differential entropies as [8] log…”

“…The concept of entropy power as defined by Shannon [7] is presented in Section 4, and the ordering of mutual information as a signal is passed through a cascaded signal processing system is stated in Section 5. The recently proposed quantity, the log ratio of entropy powers, is given in Section 6, where it is shown that the mean squared estimation errors can be substituted for the entropy powers in the ratio for an important set of probability densities [5,8,9]. The mutual information between the input speech and the short-term prediction sequence is discussed in Section 7 and the mutual information provided by the adaptive and fixed codebooks about the input speech is developed in Section 8.…”