A note on stochastic methods in connection with approximation theorems for positive linear operators

Hahn, L.

doi:10.2140/pjm.1982.101.307

Cited by 16 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note: In approximation theory literature, researchers have explored the interactions between general positive linear operator approximation and its probabilistic counterpart decades ago [66]- [68]. However, in statistics literature related to positive linear approximation, usually only specific operators are used, such as the Bernstein operator [69], and the focus may not be on obtaining the tightest bound on bias [70], [71].…”

Section: B Biasmentioning

confidence: 99%

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Jiao

Venkat

Han

et al. 2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We consider the problem of estimating functionals of discrete distributions, and focus on tight nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations, and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional, namely their \emph{bias}. We characterize the worst case squared error risk incurred by the Maximum Likelihood Estimator (MLE) in estimating the Shannon entropy $H(P) = \sum_{i = 1}^S -p_i \ln p_i$, and $F_\alpha(P) = \sum_{i = 1}^S p_i^\alpha,\alpha>0$, up to multiplicative constants, for any alphabet size $S\leq \infty$ and sample size $n$ for which the risk may vanish. As a corollary, for Shannon entropy estimation, we show that it is necessary and sufficient to have $n \gg S$ observations for the MLE to be consistent. In addition, we establish that it is necessary and sufficient to consider $n \gg S^{1/\alpha}$ samples for the MLE to consistently estimate $F_\alpha(P), 0<\alpha<1$. The minimax rate-optimal estimators for both problems require $S/\ln S$ and $S^{1/\alpha}/\ln S$ samples, which implies that the MLE has a strictly sub-optimal sample complexity. When $1<\alpha<3/2$, we show that the worst-case squared error rate of convergence for the MLE is $n^{-2(\alpha-1)}$ for infinite alphabet size, while the minimax squared error rate is $(n\ln n)^{-2(\alpha-1)}$. When $\alpha\geq 3/2$, the MLE achieves the minimax optimal rate $n^{-1}$ regardless of the alphabet size. As an application of the general theory, we analyze the Dirichlet prior smoothing techniques for Shannon entropy estimation. We show that no matter how we tune the parameters in the Dirichlet prior, this technique cannot achieve the minimax rates in entropy estimation.Comment: 27 pages, 1 figure, published in IEEE Transactions on Information Theor

show abstract

Section: B Biasmentioning

confidence: 99%

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Jiao

Venkat

Han

et al. 2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

Section: B Biasmentioning

confidence: 99%

Does dirichlet prior smoothing solve the Shannon entropy estimation problem?

Han

Jiao

Weissman

2015

2015 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Abstract-We consider the problem of estimating functionals of discrete distributions, and focus on tight (up to universal multiplicative constants for each specific functional) nonasymptotic analysis of the worst case squared error risk of widely used estimators. We apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations, and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional, namely their bias.We explicitly characterize the worst case squared error risk incurred by the Maximum Likelihood Estimator (MLE) in estimating the Shannon entropy H(P ) = S i=1 −pi ln pi, and the power sum Fα(P ) = S i=1 p α i , α > 0, up to universal multiplicative constants for each fixed functional, for any alphabet size S ≤ ∞ and sample size n for which the risk may vanish. As a corollary, for Shannon entropy estimation, we show that it is necessary and sufficient to have n ≫ S observations for the MLE to be consistent. In addition, we establish that it is necessary and sufficient to consider n ≫ S 1/α samples for the MLE to consistently estimate Fα(P ), 0 < α < 1. The minimax rate-optimal estimators for both problems require S/ ln S and S 1/α / ln S samples, which implies that the MLE has a strictly sub-optimal sample complexity. When 1 < α < 3/2, we show that the worst-case squared error rate of convergence for the MLE is n −2(α−1) for infinite alphabet size, while the minimax squared error rate is (n ln n) −2(α−1) . When α ≥ 3/2, the MLE achieves the minimax optimal rate n −1 regardless of the alphabet size.As an application of the general theory, we analyze the Dirichlet prior smoothing techniques for Shannon entropy estimation. In this context, one approach is to plug-in the Dirichlet prior smoothed distribution into the entropy functional, while the other one is to calculate the Bayes estimator for entropy under the Dirichlet prior for squared error, which is the conditional expectation. We show that in general such estimators do not improve over the maximum likelihood estimator. No matter how we tune the parameters in the Dirichlet prior, this approach cannot achieve the minimax rates in entropy estimation. The performance of the minimax rate-optimal estimator with n samples is essentially at least as good as that of Dirichlet smoothed entropy estimators with n ln n samples.

show abstract

Lipschitz–Nikolskii Constants for the Trotter–Feller Operator

Zhou

1996

Journal of Mathematical Analysis and Applications

View full text Add to dashboard Cite

In this paper we determine the Lipschitz᎐Nikolskiı constants for thȇ Trotter᎐Feller operator which contains various well-known operators. As corollaries to our general settings we obtain the Lipschitz᎐Nikolskiı constants for a series of concrete operators including the Bernstein, Szasz, Gamma, Weierstrass, and Baskakov operators, their generalizations such as the Cheney᎐Sharma and Bleimann᎐Butzer᎐Hahn operators, and many others. Our results also improve Rathore's on the Meyer-Konig and Zeller operator and the Gamma operator of Muller as well as Rathore and Singh's on the Post᎐Widder operator. Throughouẗ the paper the probabilistic method is used intensively while a result in probability theory on normal approximation plays a key role. ᮊ

show abstract

A note on stochastic methods in connection with approximation theorems for positive linear operators

Cited by 16 publications

References 7 publications

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Maximum Likelihood Estimation of Functionals of Discrete Distributions

Does dirichlet prior smoothing solve the Shannon entropy estimation problem?

Lipschitz–Nikolskii Constants for the Trotter–Feller Operator

Contact Info

Product

Resources

About