Shannon entropy estimation from convergence results in the countable alphabet case

Abstract

In this paper new results for the Shannon entropy estimation and estimation of distributions, consistently in information divergence, are presented in the countable alphabet case. Sufficient conditions for the entropy convergence are adopted, including scenarios with both finitely and infinitely supported distributions. From this approach, new estimates, strong consistency results and rate of convergences are derived for various plug-in histogram-based schemes.