Sylvester Normalizing Flows for Variational Inference

Berg, Rianne van den; Hasenclever, Leonard; Tomczak, Jakub M.; Welling, Max

doi:10.48550/arxiv.1803.05649

Cited by 14 publications

(28 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with familiar generative models, such as VAE and GAN, normalizing flows represent another generative family that has existed for a long time but only becomes popular in recent years. Normalizing flows have many types of implementation, such as planar flows [35], [36], autoregressive models [37], [38], coupling-based flows [39], [40], [41], and continuous flows [42], [43]. Dinh et al [39] introduced a coupling method to enable highly expressive transformations for flows, and this idea is further improved in [40], [41], [44].…”

Section: Normalizing Flows In Point Cloud Learningmentioning

confidence: 99%

PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows

Mao¹,

Du²,

Hou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Point cloud upsampling aims to generate dense point clouds from given sparse ones, which is a challenging task due to the irregular and unordered nature of point sets. To address this issue, we present a novel deep learning-based model, called PU-Flow, which incorporates normalizing flows and feature interpolation techniques to produce dense points uniformly distributed on the underlying surface. Specifically, we formulate the upsampling process as point interpolation in a latent space, where the interpolation weights are adaptively learned from local geometric context, and exploit the invertible characteristics of normalizing flows to transform points between Euclidean and latent spaces. We evaluate PU-Flow on a wide range of 3D models with sharp features and high-frequency details. Qualitative and quantitative results show that our method outperforms state-of-the-art deep learning-based approaches in terms of reconstruction quality, proximity-to-surface accuracy, and computation efficiency.

show abstract

Section: Normalizing Flows In Point Cloud Learningmentioning

confidence: 99%

PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows

Mao¹,

Du²,

Hou³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Good performance using normalizing flows requires that the mapping f θ be powerful yet invertible with a Jacobian that can be computed efficiently. There are a number of techniques in the literature to achieve this, e.g., linear mappings, planar/radial flows (Rezende & Mohamed, 2015;Tabak & Turner, 2013), Sylvester flows (Berg et al, 2018), coupling (Dinh et al, 2014) and auto-regressive models (Larochelle & Murray, 2011). One may also compose the transformations, e.g., using monotonic mappings f θ in each layer (Huang et al, 2018;De Cao et al, 2019).…”

Section: Background and Related Workmentioning

confidence: 99%

TraDE: Transformers for Density Estimation

Fakoor,

Chaudhari,

Mueller

et al. 2020

Preprint

View full text Add to dashboard Cite

We present TraDE, an attention-based architecture for auto-regressive density estimation. In addition to a Maximum Likelihood loss we employ a Maximum Mean Discrepancy (MMD) two-sample loss to ensure that samples from the estimate resemble the training data. The use of attention means that the model need not retain conditional sufficient statistics during the process beyond what is needed for each covariate. TraDE performs significantly better than existing approaches such differentiable flow based estimators on standard tabular and image-based benchmarks in terms of the log-likelihood on held out data. TraDE works well wide range of tasks that includes classification methods to ascertain the quality of generated samples, out of distribution sample detection, and handling outliers in the training data.

show abstract

“…There Finally, there are also several works which develop new techniques for constructing NFs that are orthogonal to ours (Tomczak & Welling, 2017;Gemici et al, 2016;Duvenaud et al, 2016;Berg et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Neural Autoregressive Flows

Huang¹,

Krueger²,

Lacoste³

et al. 2018

Preprint

View full text Add to dashboard Cite

Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF) (Papamakarios et al., 2017), and to accelerate stateof-the-art WaveNet-based speech synthesis to 20x faster than real-time (Oord et al., 2017), via Inverse Autoregressive Flows (IAF) (Kingma et al., 2016). We unify and generalize these approaches, replacing the (conditionally) affine univariate transformations of MAF/IAF with a more general class of invertible univariate transformations expressed as monotonic neural networks. We demonstrate that the proposed neural autoregressive flows (NAF) are universal approximators for continuous probability distributions, and their greater expressivity allows them to better capture multimodal target distributions. Experimentally, NAF yields state-of-the-art performance on a suite of density estimation tasks and outperforms IAF in variational autoencoders trained on binarized MNIST. 1

show abstract

Sylvester Normalizing Flows for Variational Inference

Cited by 14 publications

References 0 publications

PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows

PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows

TraDE: Transformers for Density Estimation

Neural Autoregressive Flows

Contact Info

Product

Resources

About