A Bayesian Nonparametric Estimation to Entropy

Al-Labadi, Luai; Patel, Viskakh; Vakiloroayaei, Kasra; Wan, Clement

doi:10.48550/arxiv.1903.00655

Cited by 2 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This difficulty can be avoided by setting an appropriate value of a as otherwise the prior G will become too influential. Al-Labadi et al (2019d) suggested to choose a to be at most n/2. For estimation purposes, from (5),…”

Section: Mutual Information Estimationmentioning

confidence: 99%

“…See for example, Vasicek (1976), Ebrahimi et al (1994), Alizadeh Noughabi (2010), Alizadeh Noughabi and Alizadeh Noughabi (2013) and Al-Omari (2014. Also, Al-Labadi et al (2019d) proposed an efficient Bayesian counterpart of Vasicek's estimator. For the multivariate (joint) entropy estimation, some frequentist procedures have been offered in the literature; see, for instance, Kozachenko and Leonenko (1987), Misra et al (2010), Sricharan and Hero (2012), Sricharan et al (2013), Gao et al (2016), Berrett et al (2019a), Ba and Lo (2019) and the references therein.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Test for Independence Via Bayesian Nonparametric Estimation of Mutual Information

Al-Labadi¹,

Asl²,

Saberi³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Mutual information is a well-known tool to measure the mutual dependence between variables. In this paper, a Bayesian nonparametric estimation of mutual information is established by means of the Dirichlet process and the k-nearest neighbor distance. As a direct outcome of the estimation, an easy-to-implement test of independence is introduced through the relative belief ratio. Several theoretical properties of the approach are presented. The procedure is investigated through various examples where the results are compared to its frequentist counterpart and demonstrate a good performance.

show abstract

Section: Mutual Information Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Test for Independence Via Bayesian Nonparametric Estimation of Mutual Information

Al-Labadi¹,

Asl²,

Saberi³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Kullback-Leibler Divergence for Bayesian Nonparametric Model Checking

Al-Labadi¹,

Patel²,

Vakiloroayaei³

et al. 2019

Preprint

View full text Add to dashboard Cite

Bayesian nonparametric statistics is an area of considerable research interest. While recently there has been an extensive concentration in developing Bayesian nonparametric procedures for model checking, the use of the Dirichlet process, in its simplest form, along with the Kullback-Leibler divergence is still an open problem. This is mainly attributed to the discreteness property of the Dirichlet process and that the Kullback-Leibler divergence between any discrete distribution and any continuous distribution is infinity. The approach proposed in this paper, which is based on incorporating the Dirichlet process, the Kullback-Leibler divergence and the relative belief ratio, is considered the first concrete solution to this issue. Applying the approach is simple and does not require obtaining a closed form of the relative belief ratio. A Monte Carlo study and real data examples show that the developed approach exhibits excellent performance.

show abstract

A Bayesian Nonparametric Estimation to Entropy

Cited by 2 publications

References 29 publications

A Test for Independence Via Bayesian Nonparametric Estimation of Mutual Information

A Test for Independence Via Bayesian Nonparametric Estimation of Mutual Information

Kullback-Leibler Divergence for Bayesian Nonparametric Model Checking

Contact Info

Product

Resources

About