Arbitrarily Tight Bounds on Differential Entropy of Gaussian Mixtures

Moshksar, Kamyar; Khandani, Amir K.

doi:10.1109/tit.2016.2553147

Cited by 23 publications

(26 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Herein, s m denotes the m-th symbol from M -ary constellation diagram and ρ i is the transmit power for i-th user. By using the tight approximation derived for Gaussian mixtures in [21], entropy of y i can be bounded as (γ + α N,N ) log 2 e + log 2 σ ≤ h(y i ) ≤ (γ + β N,N ) log 2 e + log 2 σ. Due to space constraints the detailed explanation on the parameters γ, α N,N and β N,N are not given here and can be found in [21].…”

Section: Achievable Ratementioning

confidence: 99%

Hybrid beamforming with spatial modulation in multi-user massive MIMO mmWave networks

Yüzgeçcioğlu

Jorswieck

2017

2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)

View full text Add to dashboard Cite

The cost of radio frequency (RF) chains is the biggest drawback of massive MIMO millimeter wave networks. By employing spatial modulation (SM), it is possible to implement lower number of RF chains than transmit antennas but still achieve high spectral efficiency. In this work, we propose a system model of the SM scheme together with hybrid beamforming at the transmitter and digital combining at the receiver. In the proposed model, spatially-modulated bits are mapped onto indices of antenna arrays. It is shown that the proposed model achieves approximately 5dB gain over classical multi-user SM scheme with only 8 transmit antennas at each antenna array. This gain can be improved further by increasing the number of transmit antennas at each array without increasing the number of RF chains.

show abstract

Section: Achievable Ratementioning

confidence: 99%

Hybrid beamforming with spatial modulation in multi-user massive MIMO mmWave networks

Yüzgeçcioğlu

Jorswieck

2017

2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)

View full text Add to dashboard Cite

show abstract

“…Note that this bound (called the Maximum Entropy Upper Bound in [13], MEUB) is tight when the GMM approximates a single Gaussian. It is fast to compute compared to the bound reported in [9] that uses Taylor' s expansion of the log-sum of the mixture density.…”

Section: Without Loss Of Generality Consider Gmms In the Formmentioning

confidence: 99%

“…Again, the Bregman divergence B F * (α : α ) is necessarily finite but KL(m(x) : m (x)) between mixtures may be potentially infinite when the KL integral diverges (hence, the condition on Jeffreys divergence finiteness). Interestingly, this Shannon information can be arbitrarily closely approximated when considering isotropic Gaussians [13]. Notice that the convex conjugate F(θ) of the continuous Shannon neg-entropy F * (η) is the log-sum-exp function on the inverse soft map.…”

Section: Appendix C On the Approximation Of Kl Between Smooth Mixturmentioning

confidence: 99%

“…We refer to [9][10][11][12] for the current state-of-the-art approximation techniques and bounds on the KL of GMMs. The latest work for computing the entropy of GMMs is [13]. It considers arbitrary finely tuned bounds of the entropy of isotropic Gaussian mixtures (a case encountered when dealing with KDEs, kernel density estimators).…”

Section: Introductionmentioning

confidence: 99%

“…It considers arbitrary finely tuned bounds of the entropy of isotropic Gaussian mixtures (a case encountered when dealing with KDEs, kernel density estimators). However, there is a catch in the technique of [13]: It relies on solving the unique roots of some log-sum-exp equations (See Theorem 1 of [13], p. 3342) that do not admit a closed-form solution. Thus it is a hybrid method that contrasts with our combinatorial approach.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities

Nielsen

Sun

2016

Entropy

View full text Add to dashboard Cite

Information-theoretic measures, such as the entropy, the cross-entropy and the Kullback-Leibler divergence between two mixture models, are core primitives in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using costly Monte Carlo stochastic integration, approximated or bounded using various techniques. We present a fast and generic method that builds algorithmically closed-form lower and upper bounds on the entropy, the cross-entropy, the Kullback-Leibler and the α-divergences of mixtures. We illustrate the versatile method by reporting our experiments for approximating the Kullback-Leibler and the α-divergences between univariate exponential mixtures, Gaussian mixtures, Rayleigh mixtures and Gamma mixtures.

show abstract

On the capacity of additive white mixture Gaussian noise channels

Sheikh‐Hosseini

Hodtani

2019

Trans Emerging Tel Tech

View full text Add to dashboard Cite

This paper provides an information‐theoretic study for the additive white impulsive noise channels by considering Gaussian mixture (GM) noise model, which includes Middleton class A and ε‐mixture Gaussian models. First, three lower and two upper bounds are found for the differential entropy of complex multivariate GM distribution and these bounds are extended to the univariate GM noise of flat fading impulsive noise channels. Second, the instantaneous capacity of these channels is bounded using two lower and two upper bounds, and the ergodic capacity and outage probability of these bounds are investigated for the log‐normal fading model. Finally, the obtained bounds are numerically illustrated and their tightness is discussed for various realizations of the GM noise. The results demonstrate that the tight bounds are not the same for all considered realizations, but they are close enough and converge to each other for a specific realization.

show abstract

Arbitrarily Tight Bounds on Differential Entropy of Gaussian Mixtures

Cited by 23 publications

References 16 publications

Hybrid beamforming with spatial modulation in multi-user massive MIMO mmWave networks

Hybrid beamforming with spatial modulation in multi-user massive MIMO mmWave networks

Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities

On the capacity of additive white mixture Gaussian noise channels

Contact Info

Product

Resources

About