Minimax optimal estimators for additive scalar functionals of discrete distributions

Abstract

In this paper, we consider estimators for an additive functional of φ, which is defined as θ(P ; φ) = k i=1 φ(pi), from n i.i.d. random samples drawn from a discrete distribution P = (p1, ..., p k ) with alphabet size k. We propose a minimax optimal estimator for the estimation problem of the additive functional. We reveal that the minimax optimal rate is characterized by the divergence speed of the fourth derivative of φ if the divergence speed is high. As a result, we show there is no consistent estimator if the divergence speed of the fourth derivative of φ is larger than p −4 . Furthermore, if the divergence speed of the fourth derivative of φ is p 4−α for α ∈ (0, 1), the minimax optimal rate is obtained within a universal multiplicative constant as n .

Minimax Optimal Additive Functional Estimation with Discrete Distribution: Slow Divergence Speed Case

“…, we have φ (1) (p) > 0 for p ∈ (0, p 1 ). From continuity of φ (1) , φ (1) (p) has the same sign in p ∈ (0, p 1 ]. Thus, if β + δ is small enough so that β + δ < p 1 ∧ 1 2 , we have…”

“…The first term is improved from Eq. (1). It indicates that the introduced estimator can consistently estimate Shannon entropy even when k n, as long as n k/ ln k. While the recent efforts revealed the minimax optimal estimators for the additive functionals with some specific φ, there is no unified methodology to derive the minimax optimal estimator for the additive functional with general φ. Jiao et al [11] suggested that their proposed estimator can be extended for general additive functional θ.…”

“…From Lemma 2, ω 2 (φ, t) tω 1 (φ (1) , t). By definition, ω 1 (φ (1) , t) t α−1 because of the Hölder continuity of φ (1) from Lemma 10.…”

“…We derive a lower bound on Eq. (9) by dividing α into three cases; α ∈ (0, 1), α = 1, and α ∈ (1,2]. For the following analysis, it is worthy to note that we have φ (2) (p) > 0 for p ∈ (0, p 0 ) where p 0 = 1 ∧ (c ′ 2 /W 2 ) 1/(α−2) because of the divergence speed assumption.…”

“…This paper was presented in part at the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany [1] and 2018 IEEE International Symposium on Information Theory (ISIT), Vail, USA [2]. Since no estimator achieves smaller worst case risk than the minimax risk, we can say that the minimax optimal estimator is the best regarding the worst case risk.…”