Dimension-Free Empirical Entropy Estimation

Cohen, Doron; Kontorovich, Aryeh; Koolyk, Aaron; Wolfer, Geoffrey

doi:10.1109/tit.2022.3232739

Cited by 2 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible further step of action may be to investigate possible other values of α on the estimation. Further work include the potential probability representation of quantum states that can be characterised by the priors considered in this paper [16], as well as future interest in dimension-free estimation of entropy in relevant settings [17].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Tsallis and Other Generalised Entropy Forms Subject to Dirichlet Mixture Priors

2022

View full text Add to dashboard Cite

Entropy indicates a measure of information contained in a complex system, and its estimation continues to receive ongoing focus in the case of multivariate data, particularly that on the unit simplex. Oftentimes the Dirichlet distribution is employed as choice of prior in a Bayesian framework conjugate to the popular multinomial likelihood with K distinct classes, where consideration of Shannon- and Tsallis entropy is of interest for insight detection within the data on the simplex. However, this prior choice only accounts for negatively correlated data, therefore this paper incorporates previously unconsidered mixtures of Dirichlet distributions as potential priors for the multinomial likelihood which addresses the drawback of negative correlation. The power sum functional, as the product moment of the mixture of Dirichlet distributions, is of direct interest in the multivariate case to conveniently access the Tsallis- and other generalized entropies that is incorporated within an estimation perspective of the posterior distribution using real economic data. A prior selection method is implemented to suggest a suitable prior for the consideration of the practitioner; empowering the user in future for consideration of suitable priors incorporating entropy within the estimation environment as well as having the option of certain mixture of Dirichlet distributions that may require positive correlation.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Using the complete product moments derived in (17), the Bayesian estimator for the power sum ( 4) can be derived by setting k i = α with i = 1, . .…”

Section: For the Flexible Dirichlet Priormentioning

confidence: 99%

Tsallis and Other Generalised Entropy Forms Subject to Dirichlet Mixture Priors

2022

View full text Add to dashboard Cite

show abstract

“…The precise asymptotic minimax rates (including the leading constant) for a fixed k and as n increases are known for 1 distance [12], 2 distance [14, p. 349], scaled 2 losses and chi-squared type distances [18,19,12], as well as KL divergence [5]. Distribution-specific rates are also known for 1 distance [11,6,7]. In terms of KL(p p(X n )), the rate of convergence for the Laplace estimator is given by max…”

Section: Introductionmentioning

confidence: 99%