Laplace Approximation for Logistic Gaussian Process Density Estimation and Regression

Abstract

Logistic Gaussian process (LGP) priors provide a flexible alternative for modelling unknown densities. The smoothness properties of the density estimates can be controlled through the prior covariance structure of the LGP, but the challenge is the analytically intractable inference. In this paper, we present approximate Bayesian inference for LGP density estimation in a grid using Laplace's method to integrate over the non-Gaussian posterior distribution of latent function values and to determine the covariance function parameters with type-II maximum a posteriori (MAP) estimation. We demonstrate that Laplace's method with MAP is sufficiently fast for practical interactive visualisation of 1D and 2D densities. Our experiments with simulated and real 1D data sets show that the estimation accuracy is close to a Markov chain Monte Carlo approximation and state-ofthe-art hierarchical infinite Gaussian mixture models. We also construct a reduced-rank approximation to speed up the computations for dense 2D grids, and demonstrate density regression with the proposed Laplace approach.

bsamGP: An R Package for Bayesian Spectral Analysis Models Using Gaussian Process Priors

“…The semiparametric density model consists of a parametric component, specified by an exponential family, and a nonparametric component, specified by a logistic Gaussian process (see, e.g., Leonard 1978and Lenk 1988. The logistic Gaussian process, defined as the logistic transformation of a Gaussian process, provides a flexible model for estimating unknown densities using the covariance structure (see, e.g., Lenk 1991;Tokdar and Ghosh 2007;Riihimäki and Vehtari 2014).…”

A conditional density estimation partition model using logistic Gaussian processes

“…However, due to lack of conjugacy, p(y i | θ i , Z i , T ) is not available explicitly. Therefore, we use a Laplace approximation [Riihimäki et al, 2014] to integrate over f i in each partition element. The posterior approximation of f i takes the following form in each partition…”

“…where σ 2 i is the magnitude hyperparameter and l i is a length-scale hyperparameter which together govern the smoothness properties of f i . We place a weakly informative half Student-t distribution with one degree of freedom and a variance equal to 10 for the magnitude parameter and the same prior with a variance of 1 for the length-scale hyperparameter [Riihimäki et al, 2014]. We then obtain the maximum a posteriori (MAP) estimate of the posterior mode of θ i using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton algorithm and set θ i equal to this value.…”

Estimation of the intensity function of an inhomogeneous Poisson process with a change‐point

“…To make inference more tractable, we discretize the interval [0,1] into m equally spaced sub‐intervals. The use of grid points is a popular approach in Gaussian process density estimation (Tokdar & Ghosh, ; Riihimäki & Vehtari, ). Because of the discretization, the computational complexity is independent of the number of observations.…”