Reparameterizing Distributions on Lie Groups

Falorsi, Luca; Haan, Pim de; Davidson, Tim R.; Forré, Patrick

doi:10.48550/arxiv.1903.02958

Cited by 9 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two models that have notable similarities to ours are the Lie VAE [14] and the Manifold Autoencoder [23]. Both models also use Lie group representations of transformations in the latent space.…”

Section: Table 1: Comparison Of Vae Techniquesmentioning

confidence: 85%

“…No No R-VAE [19] Yes Nonlinear No Lie VAE [14] No Nonlinear No VAELLS (our approach) Yes Nonlinear Yes…”

Section: Modelmentioning

confidence: 99%

“…A mismatch between the latent prior distribution of a VAE and the data manifold structure can lead to over regularization and poor data representation. Other models incorporate more complex latent structures such as hyperspheres [12], tori [13,14], and hyperboloids [15,16,17]. These models have demonstrated their suitability for certain datasets by choosing a prior that is the best match out of a predefined set of candidates.…”

Section: Table 1: Comparison Of Vae Techniquesmentioning

confidence: 99%

See 2 more Smart Citations

Variational Autoencoder with Learned Latent Structure

Connor,

Canal,

Rozell

2020

Preprint

View full text Add to dashboard Cite

The manifold hypothesis states that high-dimensional data can be modeled as lying on or near a low-dimensional, nonlinear manifold. Variational Autoencoders (VAEs) approximate this manifold by learning mappings from low-dimensional latent vectors to high-dimensional data while encouraging a global structure in the latent space through the use of a specified prior distribution. When this prior does not match the structure of the true data manifold, it can lead to a less accurate model of the data. To resolve this mismatch, we introduce the Variational Autoencoder with Learned Latent Structure (VAELLS) which incorporates a learnable manifold model into the latent space of a VAE. This enables us to learn the nonlinear manifold structure from the data and use that structure to define a prior in the latent space. The integration of a latent manifold model not only ensures that our prior is well-matched to the data, but also allows us to define generative transformation paths in the latent space and describe class manifolds by transformations stemming from examples of each class. We validate our model on examples with known latent structure and also demonstrate its capabilities on a real-world dataset. 1

show abstract

Section: Table 1: Comparison Of Vae Techniquesmentioning

confidence: 85%

“…No No R-VAE [19] Yes Nonlinear No Lie VAE [14] No Nonlinear No VAELLS (our approach) Yes Nonlinear Yes…”

Section: Modelmentioning

confidence: 99%

Section: Table 1: Comparison Of Vae Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Variational Autoencoder with Learned Latent Structure

Connor,

Canal,

Rozell

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We believe that FNPs open the door to plenty of exciting avenues for future research; designing better function priors by e.g. imposing a manifold structure on the FNP latents [12], extending FNPs to unsupervised learning by e.g. adapting ACNs [16] or considering hierarchical models similar to deep GPs [10].…”

Section: Discussionmentioning

confidence: 99%

The Functional Neural Process

Louizos,

Shi,

Schutte

et al. 2019

Preprint

View full text Add to dashboard Cite

We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs). FNPs model distributions over functions by learning a graph of dependencies on top of latent representations of the points in the given dataset. In doing so, they define a Bayesian model without explicitly positing a prior distribution over latent global parameters; they instead adopt priors over the relational structure of the given dataset, a task that is much simpler. We show how we can learn such models from data, demonstrate that they are scalable to large datasets through mini-batch optimization and describe how we can make predictions for new points via their posterior predictive distribution. We experimentally evaluate FNPs on the tasks of toy regression and image classification and show that, when compared to baselines that employ global latent parameters, they offer both competitive predictions as well as more robust uncertainty estimates.Preprint. Under review.

show abstract

“…This leads to so called wrapped distributions P W θ = exp x P , with P a probability measure on R d . This approach has been, for instance, been taken by Falorsi et al (2019) and Bose et al (2020) to parametrize probability distributions on Lie groups and hyperbolic space. However, projected methods based on the exponential map often lead to numerical and computational challenges.…”

Section: Curved Surfacesmentioning

confidence: 99%

Moser Flow: Divergence-based Generative Modeling on Manifolds

Rozen¹,

Grover²,

Nickel³

et al. 2021

Preprint

View full text Add to dashboard Cite

We are interested in learning generative models for complex geometries described via manifolds, such as spheres, tori, and other implicit surfaces. Current extensions of existing (Euclidean) generative models are restricted to specific geometries and typically suffer from high computational costs. We introduce Moser Flow (MF), a new class of generative models within the family of continuous normalizing flows (CNF). MF also produces a CNF via a solution to the change-of-variable formula, however differently from other CNF methods, its model (learned) density is parameterized as the source (prior) density minus the divergence of a neural network (NN). The divergence is a local, linear differential operator, easy to approximate and calculate on manifolds. Therefore, unlike other CNFs, MF does not require invoking or backpropagating through an ODE solver during training. Furthermore, representing the model density explicitly as the divergence of a NN rather than as a solution of an ODE facilitates learning high fidelity densities. Theoretically, we prove that MF constitutes a universal density approximator under suitable assumptions. Empirically, we demonstrate for the first time the use of flow models for sampling from general curved surfaces and achieve significant improvements in density estimation, sample quality, and training complexity over existing CNFs on challenging synthetic geometries and real-world benchmarks from the earth and climate sciences.Preprint. Under review.

show abstract

Reparameterizing Distributions on Lie Groups

Cited by 9 publications

References 0 publications

Variational Autoencoder with Learned Latent Structure

Variational Autoencoder with Learned Latent Structure

The Functional Neural Process

Moser Flow: Divergence-based Generative Modeling on Manifolds

Contact Info

Product

Resources

About