The Functional Neural Process

Louizos, Christos; Shi, Xiahan; Schutte, Klamer; Welling, Max

doi:10.48550/arxiv.1906.08324

Cited by 5 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we demonstrated the performance of the proposed priors on a 1D test function, as used in [Louizos et al, 2019]. We compare to BNN with independent Gaussian priors and a mean-field Gaussian variational approximation, and MetaNN [Karaletsos et al, 2018].…”

Section: B2 Results On a Synthetic Regression Examplementioning

confidence: 99%

Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights

Karaletsos,

Bui

2020

Preprint

View full text Add to dashboard Cite

Probabilistic neural networks are typically modeled with independent weight priors, which do not capture weight correlations in the prior and do not provide a parsimonious interface to express properties in function space. A desirable class of priors would represent weights compactly, capture correlations between weights, facilitate calibrated reasoning about uncertainty, and allow inclusion of prior knowledge about the function space such as periodicity or dependence on contexts such as inputs. To this end, this paper introduces two innovations: (i) a Gaussian process-based hierarchical model for network weights based on unit embeddings that can flexibly encode correlated weight structures, and (ii) input-dependent versions of these weight priors that can provide convenient ways to regularize the function space through the use of kernels defined on contextual inputs. We show these models provide desirable test-time uncertainty estimates on out-of-distribution data, demonstrate cases of modeling inductive biases for neural networks with kernels which help both interpolation and extrapolation from training data, and demonstrate competitive predictive performance on an active learning benchmark.

show abstract

Section: B2 Results On a Synthetic Regression Examplementioning

confidence: 99%

Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights

Karaletsos,

Bui

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…On the inference side, there are attentive NPs , which endow the encoder with self-attention (and thus make it Turing complete, Pérez et al, 2021), and convolutional (conditional) NPs (Gordon et al, 2019;Foong et al, 2020), which add translation equivariance to the model. On the generative side, there are functional NPs (Louizos et al, 2019), which introduce dependence between the predictions by learning a relational graph structure over the latents z and Gaussian NPs (Bruinsma et al, 2021), which achieve a similar property by replacing the generative likelihood with a GP, the mean and kernel of which are inferred based on the latents.…”

Section: Neural Processesmentioning

confidence: 99%

Priors in Bayesian Deep Learning: A Review

Fortuin

2022

Int Statistical Rev

View full text Add to dashboard Cite

While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians. In this review, we highlight the importance of prior choices for Bayesian deep learning and present an overview of different priors that have been proposed for (deep) Gaussian processes, variational autoencoders and Bayesian neural networks. We also outline different methods of learning priors for these models from data. We hope to motivate practitioners in Bayesian deep learning to think more carefully about the prior specification for their models and to provide them with some inspiration in this regard.

show abstract

“…NGPs (Neural gaussian Process) [11,12,18] are novel class of probabilistic neural models for accurate and calibrated predictions. NGP models such as EPIFNP [12] and its variants [11] have shown state-of-art performance in calibrated probabilistic forecasting across multiple benchmarks for single time-series.…”

Section: Raw Forecast Distributions From Ngpsmentioning

confidence: 99%

PROFHIT: Probabilistic Robust Forecasting for Hierarchical Time-series

Kamarthi¹,

Kong²,

Rodríguez³

et al. 2022

Preprint

View full text Add to dashboard Cite

Probabilistic hierarchical time-series forecasting is an important variant of timeseries forecasting, where the goal is to model and forecast multivariate timeseries that have underlying hierarchical relations. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. Recent state-of-art probabilistic forecasting methods also impose hierarchical relations on point predictions and samples of distribution which does not account for coherency of forecast distributions. Previous works also silently assume that datasets are always consistent with given hierarchical relations and do not adapt to real-world datasets that show deviation from this assumption. We close both these gap and propose PROFHIT, which is a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy. PROFHIT uses a flexible probabilistic Bayesian approach and introduces a novel Distributional Coherency regularization to learn from hierarchical relations for entire forecast distribution that enables robust and calibrated forecasts as well as adapt to datasets of varying hierarchical consistency. On evaluating PROFHIT over wide range of datasets, we observed 41-88% better performance in accuracy and calibration. Due to modeling the coherency over full distribution, we observed that PROFHIT can robustly provide reliable forecasts even if up to 10% of input time-series data is missing where other methods' performance severely degrade by over 70%.Preprint. Under review.

show abstract

The Functional Neural Process

Cited by 5 publications

References 18 publications

Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights

Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights

Priors in Bayesian Deep Learning: A Review

PROFHIT: Probabilistic Robust Forecasting for Hierarchical Time-series

Contact Info

Product

Resources

About