Kullback Leibler property of kernel mixture priors in Bayesian density estimation

Abstract

Positivity of the prior probability of Kullback-Leibler neighborhood around the true density, commonly known as the Kullback-Leibler property, plays a fundamental role in posterior consistency. A popular prior for Bayesian estimation is given by a Dirichlet mixture, where the kernels are chosen depending on the sample space and the class of densities to be estimated. The Kullback-Leibler property of the Dirichlet mixture prior has been shown for some special kernels like the normal density or Bernstein polynomial, under appropriate conditions. In this paper, we obtain easily verifiable sufficient conditions, under which a prior obtained by mixing a general kernel possesses the Kullback-Leibler property. We study a wide variety of kernel used in practice, including the normal, t, histogram, gamma, Weibull densities and so on, and show that the Kullback-Leibler property holds if some easily verifiable conditions are satisfied at the true density. This gives a catalog of conditions required for the Kullback-Leibler property, which can be readily used in applications.

The L1-consistency of Dirichlet mixtures in multivariate Bayesian density estimation

“…Such a property plays a very important role in consistency. The result generalizes Theorem 5 of Wu and Ghosal [16].…”

“…By a theorem of Schwartz [11], this implies weak consistency. Note that Theorem 5 of Wu and Ghosal [16] for the Dirichlet mixture with a scaled type multivariate normal density as its kernel is a special case of Theorem 2 in this paper. In Section 4, we first state a lemma which gives bounds for metric entropies, then state another lemma which gives the sufficient conditions to satisfy Condition (A1).…”

On some aspects of the asymptotic properties of Bayesian approaches in nonparametric and semiparametric models

“…These are the same conditions as those considered in [Wu and Ghosal, 2008] in the case where the base measure of the Dirichlet process is not data dependent. Sections 1.1 and 2 concern various aspects of Bayesian nonparametric or semiparametric estimation procedures.…”

Bayesian Nonparametric Approach to Multiple Testing

“…The nonparametric nature of K can be maintained by putting a prior with large weak support, such as the Dirichlet process. A recent result of [61] shows that nonparametric Bayesian density estimation based on a skew-normal kernel is consistent under the weak topology, adding a strong justification for the use of this kernel. Interestingly, if the theoretical standard normal null distribution is way off from the empirical one, then one can incorporate this feature in the model by allowing K to assign weights to skew-normal components stochastically larger than the standard normal.…”