Hilbert-Space Reduced-Rank Methods For Deep Gaussian Processes

Emzir, Muhammad F.; Lasanen, Sari; Purisha, Zenith; Särkkä, Simo

doi:10.1109/mlsp.2019.8918874

Cited by 8 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, several hierarchical models, which promote more versatile behaviours, have been developed. These include, for example, deep Gaussian processes (Dunlop et al, 2018;Emzir et al, 2019), level-set methods (Dunlop et al, 2017), mixtures of compound Poisson processes and Gaussians (Hosseini, 2017), and stacked Matérn fields via stochastic partial differential equations (Roininen et al, 2019). The problem with hierarchical priors is that in the posteriors the parameters and hyperparameters may become strongly coupled, which means that vanilla MCMC methods become problematic and, for example, re-parameterisations are needed for sampling the posterior efficiently (Chada et al, 2019;Monterrubio-Gómez et al, 2019).…”

Section: Literature Reviewmentioning

confidence: 99%

Enhancing Industrial X-ray Tomography by Data-Centric Statistical Methods

Suuronen¹,

Emzir²,

Lasanen³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

X-ray tomography has applications in various industrial fields such as sawmill industry, oil and gas industry, chemical engineering, and geotechnical engineering. In this article, we study Bayesian methods for the X-ray tomography reconstruction. In Bayesian methods, the inverse problem of tomographic reconstruction is solved with help of a statistical prior distribution which encodes the possible internal structures by assigning probabilities for smoothness and edge distribution of the object. We compare Gaussian random field priors, that favour smoothness, to non-Gaussian total variation, Besov, and Cauchy priors which promote sharp edges and high-contrast and lowcontrast areas in the object. We also present computational schemes for solving the resulting high-dimensional Bayesian inverse problem with 100,000-1,000,000 unknowns. In particular, we study the applicability of a no-U-turn variant of Hamiltonian Monte Carlo methods and of a more classical adaptive Metropolis-within-Gibbs algorithm for this purpose. These methods also enable full uncertainty quantification of the reconstructions. For faster computations, we use maximum a posteriori estimates with limited-memory BFGS optimisation algorithm. As the first industrial application, we consider sawmill industry X-ray log tomography. The logs have knots, rotten parts, and even possibly metallic pieces, making them good examples for non-Gaussian priors. Secondly, we study drill-core rock sample tomography, an example from oil and gas industry. We show that Cauchy priors produce smaller number of artefacts than other choices, especially with sparse high-noise measurements, and choosing Hamiltonian Monte Carlo enables systematic uncertainty quantification. Impact StatementIndustrial X-ray tomography reconstruction accuracy depends on various factors, like the equipment, measurement geometry and constraints of the target. For example dynamical systems are harder targets than static ones. The harder and noisier the setting becomes, the more emphasis goes on mathematical modelling of the targets. Bayesian statistical inversion is a common choice for difficult measurement settings, and its limitations mainly come from the choice of the a priori models. Gaussian models are widely studied, but they provide smooth reconstructions. Total variation priors are not invariant under mesh changes, so doing systematic uncertainty quantification, like data-centric sensor optimisation, cannot be done with them. Besov and Cauchy priors however provide systematic non-Gaussian random field models, which can be used for contrast-boosting tomography. The drawback is higher computational cost. Hence, the techniques developed here are useful for non-time-critical applications with difficult measurement settings. In these cases, the methods developed may provide significantly better reconstructions than the traditional methods, like filtered back-projection.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Enhancing Industrial X-ray Tomography by Data-Centric Statistical Methods

Suuronen¹,

Emzir²,

Lasanen³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The idea of putting GP priors on the hyperpameters of a GP (Heinonen et al 2016;Roininen et al 2019;Salimbeni and Deisenroth 2017b) can be continued hierarchically, which leads to one type of deep Gaussian process (DGP) construction (Dunlop et al 2018;Emzir et al 2019Emzir et al , 2020. Namely, the GP is conditioned on another GP, which again depends on another GP, and so forth.…”

Section: Introductionmentioning

confidence: 99%

Deep state-space Gaussian processes

2021

View full text Add to dashboard Cite

This paper is concerned with a state-space approach to deep Gaussian process (DGP) regression. We construct the DGP by hierarchically putting transformed Gaussian process (GP) priors on the length scales and magnitudes of the next level of Gaussian processes in the hierarchy. The idea of the state-space approach is to represent the DGP as a non-linear hierarchical system of linear stochastic differential equations (SDEs), where each SDE corresponds to a conditional GP. The DGP regression problem then becomes a state estimation problem, and we can estimate the state efficiently with sequential methods by using the Markov property of the state-space DGP. The computational complexity scales linearly with respect to the number of measurements. Based on this, we formulate state-space MAP as well as Bayesian filtering and smoothing solutions to the DGP regression problem. We demonstrate the performance of the proposed models and methods on synthetic non-stationary signals and apply the state-space DGP to detection of the gravitational waves from LIGO measurements.

show abstract

“…In this work, we consider the challenge of accurate and fast computations in the presence of Gaussian processes with parameterized covariance kernels. In view of the literature on deep [7,9,12,11] and shallow [15] Gaussian processes, we refer to parameterized covariance kernels as shallow covariance kernels. Moreover, we denote the Gaussian process as outer layer and the distribution of the hyper-parameter as inner layer.…”

mentioning

confidence: 99%

“…For a given parameter θ ∈ Θ, we compute the Cholesky factor L ∈ R k×k of C(θ)(I, I). For a sample ξ ∼ N(0, Id k ) we have We now discuss the computational cost of drawing a sample from M (•|θ) via (12). First, we need to construct the matrices C(θ)(:, I), C(θ)(I, I) which comes at a cost of O(nks) if the linear expansion of C has s terms.…”

mentioning

confidence: 99%