Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities

Ghosal, Subhashis; Vaart, Aad van der

doi:10.1214/aos/1013203452

Cited by 190 publications

(238 citation statements)

References 25 publications

Supporting

Mentioning

229

Contrasting

Order By: Relevance

“…Our aim is to approximate

f_{0}

using multivariate Gaussian RBF‐net. The result essentially extends the approximation result in Lemma 3.1 of [22] from dimension 1 to dimension

p_{0}

. Thus they share commonalities in the expression of the obtained approximation.…”

Section: Theoretical Supportsupporting

confidence: 82%

Multivariate Gaussian RBF‐net for smooth function estimation and variable selection

Roy

2021

Statistical Analysis

View full text Add to dashboard Cite

Neural networks are routinely used for nonparametric regression modeling. The interest in these models is growing with ever-increasing complexities in modern datasets. With modern technological advancements, the number of predictors frequently exceeds the sample size in many application areas. Thus, selecting important predictors from the huge pool is an extremely important task for judicious inference. This paper proposes a novel flexible class of single-layer radial basis functions (RBF) networks. The proposed architecture can estimate smooth unknown regression functions and also perform variable selection. We primarily focus on Gaussian RBF-net due to its attractive properties. The extensions to other choices of RBF are fairly straightforward. The proposed architecture is also shown to be effective in identifying relevant predictors in a low-dimensional setting using the posterior samples without imposing any sparse estimation scheme. We develop an efficient Markov chain Monte Carlo algorithm to generate posterior samples of the parameters. We illustrate the proposed method's empirical efficacy through simulation experiments, both in high and low dimensional regression problems. The posterior contraction rate is established with respect to empirical 2 distance assuming that the error variance is unknown, and the true function belongs to a Hölder ball. We illustrate our method in a Human Connectome Project dataset to predict vocabulary comprehension and to identify important edges of the structural connectome.

show abstract

“…Our aim is to approximate

f_{0}

using multivariate Gaussian RBF‐net. The result essentially extends the approximation result in Lemma 3.1 of [22] from dimension 1 to dimension

p_{0}

. Thus they share commonalities in the expression of the obtained approximation.…”

Section: Theoretical Supportsupporting

confidence: 82%

Multivariate Gaussian RBF‐net for smooth function estimation and variable selection

Roy

2021

Statistical Analysis

View full text Add to dashboard Cite

show abstract

“…Starting with error terms, a possibility is to assume a flexible parametric family for errors, for example using normal mixtures. 30 Ghosal and Van der Vaart (2001 provide results on the ability of normal mixtures to approximate unknown densities. Imposing a flexible parametric structure should not be seen as a severe limitation if the conditions of the identification theorems are satisfied.…”

Section: Densitiesmentioning

confidence: 99%

Identifying Distributional Characteristics in Random Coefficients Panel Data Models

Arellano

Bonhomme

2011

The Review of Economic Studies

147

View full text Add to dashboard Cite

show abstract

“…For the Gaussian kernel and the Dirichlet Process prior of the mixing distribution, asymptotic properties, such as consistency, and rate of convergence of the posterior distribution based on kernel mixture priors were established by Ghosal, Ghosh, and Ramamoorthi (1999), Tokdar (2006), and van der Vaart (2001, 2007). Similar results for Dirichlet mixture of Bernstein polynomials were shown by Petrone and Wasserman (2002), Ghosal (2001) and Kruijer and van der Vaart (2008).…”

Section: Bayesian Nonparametric Copula Kernel Mixturementioning

confidence: 73%

“…For a subjective Bayesian who dispenses of the notion of a true parameter, consistency has an important connection with the stability of predictive distributions of future observations -a consistent posterior will tend to agree with calculations of other Bayesians using a different prior distribution in the sense of weak topology. For an objective Bayesian who assumes the existence of an unknown true model, consistency can be thought of as a validation of the Bayesian method as approaching the mechanism used to generate the data(Ghosal and van der Vaart, 2011). In this sense, it is desirable to establish the conditions under which infinite mixture models have a consistent posterior, as the use of models that are inconsistent can be regarded as ill-advised.…”

mentioning

confidence: 99%

Copula based factorization in Bayesian multivariate infinite mixture models

Burda

Prokhorov

2014

Journal of Multivariate Analysis

View full text Add to dashboard Cite

Bayesian nonparametric models based on infinite mixtures of density kernels have been recently gaining in popularity due to their flexibility and feasibility of implementation even in complicated modeling scenarios. In economics, they have been particularly useful in estimating nonparametric distributions of latent variables. However, these models have been rarely applied in more than one dimension. Indeed, the multivariate case suffers from the curse of dimensionality, with a rapidly increasing number of parameters needed to jointly characterize each mixing component. In this paper, we propose a factorization scheme for nonparametric mixture models whereby each marginal dimension in the mixing parameter space is modeled separately, linked by a nonparametric random copula function. Specifically, we consider nonparametric univariate Gaussian mixtures for the marginals and a multivariate random Bernstein polynomial copula for the link function, under Dirichlet process priors. We show that this scheme leads to an improvement in the precision of a density estimate in finite samples, providing a suitable tool for applications in higher dimensions. We derive weak posterior consistency of the copula-based mixing scheme for general kernel types under high-level conditions, and strong posterior consistency for the specific Bernstein-Gaussian mixture model.

show abstract

Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities

Cited by 190 publications

References 25 publications

Multivariate Gaussian RBF‐net for smooth function estimation and variable selection

Multivariate Gaussian RBF‐net for smooth function estimation and variable selection

Identifying Distributional Characteristics in Random Coefficients Panel Data Models

Copula based factorization in Bayesian multivariate infinite mixture models

Contact Info

Product

Resources

About