PriorVAE: encoding spatial priors with variational autoencoders for small-area estimation

Abstract

Gaussian processes (GPs), implemented through multivariate Gaussian distributions for a finite collection of data, are the most popular approach in small-area spatial statistical modelling. In this context, they are used to encode correlation structures over space and can generalize well in interpolation tasks. Despite their flexibility, off-the-shelf GPs present serious computational challenges which limit their scalability and practical usefulness in applied settings. Here, we propose a novel, deep generative modelling approach to tackle this challenge, termed PriorVAE: for a particular spatial setting, we approximate a class of GP priors through prior sampling and subsequent fitting of a variational autoencoder (VAE). Given a trained VAE, the resultant decoder allows spatial inference to become incredibly efficient due to the low dimensional, independently distributed latent Gaussian space representation of the VAE. Once trained, inference using the VAE decoder replaces the GP within a Bayesian sampling framework. This approach provides tractable and easy-to-implement means of approximately encoding spatial priors and facilitates efficient statistical inference. We demonstrate the utility of our VAE two-stage approach on Bayesian, small-area estimation tasks.

Bayesian models for event-related potentials time-series and electrode correlations estimation

“…Thus, the exploration of these more intricate models may also require the use of variational inference which is also able to acknowledge correlations within the posterior, or similar methods (e.g. Semenova et al, 2022).…”

$$\pi $$VAE: a stochastic process prior for Bayesian deep learning with MCMC

“…(Semenova et al. 2022 ) introduced a novel application of VAEs in a Bayesian inference setting, using a two stage approach that is closely related to ours. In brief, in the first stage, a VAE is trained to encode and decode a large dataset of vectors consisting of samples drawn from a specified prior over random vectors.…”

“…In a challenging spatial statistics setting, (Semenova et al. 2022 ) used this approach and achieved MCMC effective sample sizes exceeding actual sample sizes, due to the incredible efficiency of the MCMC sampler.…”

“…While this simple example proved useful in some settings (Semenova et al. 2022 ), inference and prediction is not possible at new input locations, because a VAE is not a stochastic process. As described above, a VAE provides a novel prior over random vectors.…”

“…A second key benefit is that when inference is performed, our sampler operates in a low dimensional, uncorrelated latent space which greatly aids efficiency and computation, as demonstrated in the spatial statistics setting in PriorVAE (Semenova et al. 2022 ). One limitation of this approach (and of PriorVAE) is that we are restricted to encoding finite-dimensional priors, because VAEs are not stochastic processes.…”