Minimax risk for missing mass estimation

Rajaraman, Nikhilesh; Thangaraj, Andrew; Suresh, Ananda Theertha

doi:10.1109/isit.2017.8007085

Cited by 21 publications

(14 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remarks: As we show in Section VII-A, Var(M 0 ) ≤ γ/n, and there exist uniform distributions that nearly achieve this variance upper bound [19]. Now, if Z ∼ sub-Gaussian(v), it is known that v ≥ var(Z) [24].…”

Section: B Concentration Resultsmentioning

confidence: 99%

“…Paraphrasing the first part of the theorem, the minimax risk of the order-α missing mass for a positive integer α falls as 1/n 2α−1 . In a proof presented in Section IV, we show the lower bound in (19) using Dirichlet priors and the upper bound using a generalized Good-Turing estimator for M 0,α (X n , P ), denoted M GT 0,α (X n ), defined as…”

Section: Summary Of Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Estimation and Concentration of Missing Mass of Functions of Discrete Probability Distributions

Chandra¹,

Thangaraj²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Given a positive function g from [0, 1] to the reals, the function's missing mass in a sequence of iid samples, defined as the sum of g(Pr(x)) over the missing letters x, is introduced and studied.The missing mass of a function generalizes the classical missing mass, and has several interesting connections to other related estimation problems. Minimax estimation is studied for order-α missing mass (g(p) = p α ) for both integer and non-integer values of α. Exact minimax convergence rates are obtained for the integer case. Concentration is studied for a class of functions and specific results are derived for order-α missing mass and missing Shannon entropy (g(p) = −p log p). Sub-Gaussian tail bounds with near-optimal worst-case variance factors are derived. Two new notions of concentration, named strongly sub-Gamma and filtered sub-Gaussian concentration, are introduced and shown to result in right tail bounds that are better than those obtained from sub-Gaussian concentration.Index terms-Missing mass, Good-Turing estimator, missing mass of a function, entropy, Mean squared error, minimax optimality, concentration, tail bounds, sub-Gaussian and sub-Gamma tails. I. INTRODUCTIONLet P be an arbitrary discrete distribution on an alphabet X . For x ∈ X , let p x P (x). Given a positive function g : [0, 1] → [0, ∞), we define a so-called additive function G(P ) over the distribution P as follows:

show abstract

Section: B Concentration Resultsmentioning

confidence: 99%

Section: Summary Of Resultsmentioning

confidence: 99%

Estimation and Concentration of Missing Mass of Functions of Discrete Probability Distributions

Chandra¹,

Thangaraj²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As mentioned in the introduction, a good part of the work on learning the missing mass focused on additive error [ 2 , 3 , 4 ]. Recently, minimax lower bounds were given for the additive error in [ 11 ] and [ 12 ]. Note however that relative error bounds cannot be deduced from these (nor any other way, given the impossibility established here.)…”

Section: Discussionmentioning

confidence: 99%

On the Impossibility of Learning the Missing Mass

Mossel

Ohannessian

2019

Entropy

View full text Add to dashboard Cite

This paper shows that one cannot learn the probability of rare events without imposing further structural assumptions. The event of interest is that of obtaining an outcome outside the coverage of an i.i.d. sample from a discrete distribution. The probability of this event is referred to as the "missing mass". The impossibility result can then be stated as: the missing mass is not distribution-free PAC-learnable in relative error. The proof is semi-constructive and relies on a coupling argument using a dithered geometric distribution. This result formalizes the folklore that in order to predict rare events, one necessarily needs distributions with "heavy tails".

show abstract

“…We relate the Good-Turing estimator for feature allocation models with the classical Good-Turing estimator for species sampling, by relying on the limiting Poisson approximation of Binomial random variables. In particular, Theorem 2.1 states that, in the feature allocation models, the Good-Turing estimator achieves a risk of order R n ≍ W n , while it is known from Rajaraman et al (2017) that the riskR n of the Good Turing estimator in the species sampling case asymptotically behaves as R n ≍ 1/n. In order to compare the two models, we will consider the limiting scenario when W → 0.…”

Section: Connection To the Good Turing Estimator For Species Samplingmentioning

confidence: 99%

A Good-Turing estimator for feature allocation models

Ayed¹,

Battiston²,

Camerlenghi³

et al. 2019

Electron. J. Statist.

View full text Add to dashboard Cite

Feature allocation models generalize species sampling models by allowing every observation to belong to more than one species, now called features. Under the popular Bernoulli product model for feature allocation, given n samples, we study the problem of estimating the missing mass Mn, namely the expected number hitherto unseen features that would be observed if one additional individual was sampled. This is motivated by numerous applied problems where the sampling procedure is expensive, in terms of time and/or financial resources allocated, and further samples can be only motivated by the possibility of recording new unobserved features. We introduce a simple, robust and theoretically sound nonparametric estimatorMn of Mn.Mn turns out to have the same analytic form of the popular Good-Turing estimator of the missing mass in species sampling models, with the difference that the two estimators have different ranges. We show that Mn admits a natural interpretation both as a jackknife estimator and as a nonparametric empirical Bayes estimator, we give provable guarantees for the performance ofMn in terms of minimax rate optimality, and we provide with an interesting connection betweenMn and the Good-Turing estimator for species sampling. Finally, we derive non-asymptotic confidence intervals forMn, which are easily computable and do not rely on any asymptotic approximation. Our approach is illustrated with synthetic data and SNP data from the ENCODE sequencing genome project.

show abstract

Minimax risk for missing mass estimation

Cited by 21 publications

References 19 publications

Estimation and Concentration of Missing Mass of Functions of Discrete Probability Distributions

Estimation and Concentration of Missing Mass of Functions of Discrete Probability Distributions

On the Impossibility of Learning the Missing Mass

A Good-Turing estimator for feature allocation models

Contact Info

Product

Resources

About