Glow: Generative Flow with Invertible 1x1 Convolutions

Kingma, Diederik P.; Dhariwal, Prafulla

doi:10.48550/arxiv.1807.03039

Cited by 127 publications

(203 citation statements)

References 14 publications

Supporting

Mentioning

202

Contrasting

Order By: Relevance

“…For the normalizing flow in this work, we build on the same basic architecture and training procedure as Ting & Weinberg (2021), using a batch size of 1024 and 500 epochs. This architecture consists of eight units of a "Neural Spline Flow", each of which consists of three layers of densely connected neural networks with 16 neurons, coupled with a "Conv1x1" operation (often referred to as "GLOW" in the machine learning literature; see, e.g., Kingma & Dhariwal 2018;Durkan et al 2019). See Green & Ting (2020); Ting & Weinberg (2021) for additional details, particularly Fig.…”

Section: Inferring the Population Density With Normalizing Flowsmentioning

confidence: 99%

A New Census of the 0.2 < z < 3.0 Universe. I. The Stellar Mass Function

Leja

Speagle

Johnson

et al. 2020

ApJ

112

149

View full text Add to dashboard Cite

There has been a long-standing factor-of-two tension between the observed star formation rate density and the observed stellar mass buildup after z ∼ 2. Recently we have proposed that sophisticated panchromatic SED models can resolve this tension, as these methods infer systematically higher masses and lower star formation rates than standard approaches. In a series of papers we now extend this analysis and present a complete, self-consistent census of galaxy formation over 0.2 < z < 3 inferred with the Prospector galaxy SED-fitting code. In this work, Paper I, we present the evolution of the galaxy stellar mass function using new mass measurements of ∼10 5 galaxies in the 3D-HST and COSMOS-2015 surveys. We employ a new methodology to infer the mass function from the observed stellar masses: instead of fitting independent mass functions in a series of fixed redshift intervals, we construct a continuity model that directly fits for the redshift evolution of the mass function. This approach ensures a smooth picture of galaxy assembly and makes use of the full, non-Gaussian uncertainty contours in our stellar mass inferences. The resulting mass function has higher number densities at a fixed stellar mass than almost any other measurement in the literature, largely owing to the older stellar ages inferred by Prospector. The stellar mass density is ∼50% higher than previous measurements, with the offset peaking at z ∼ 1. The next two papers in this series will present the new measurements of star-forming main sequence and the cosmic star formation rate density, respectively.

show abstract

Section: Inferring the Population Density With Normalizing Flowsmentioning

confidence: 99%

A New Census of the 0.2 < z < 3.0 Universe. I. The Stellar Mass Function

Leja

Speagle

Johnson

et al. 2020

ApJ

112

149

View full text Add to dashboard Cite

show abstract

“…is the determinant of the neural network function's Jacobian matrix. Therefore, the probability of a random sample from q θ (x) is easy to evaluate when the Jacobian matrix is computationally tractable, as is the case with commonly used normalizing flow forms includes NICE (Dinh et al 2014), Real-NVP (Dinh et al 2016), and Glow (Kingma & Dhariwal 2018). As described in Section 4.3, in this work we choose to use a Real-NVP generative network to parameterize q θ (x).…”

Section: Normalizing Flowmentioning

confidence: 99%

alpha-Deep Probabilistic Inference (alpha-DPI): efficient uncertainty quantification from exoplanet astrometry to black hole feature extraction

Sun,

Bouman,

Tiede

et al. 2022

Preprint

View full text Add to dashboard Cite

Inference is crucial in modern astronomical research, where hidden astrophysical features and patterns are often estimated from indirect and noisy measurements. Inferring the posterior of hidden features, conditioned on the observed measurements, is essential for understanding the uncertainty of results and downstream scientific interpretations. Traditional approaches for posterior estimation include sampling-based methods and variational inference. However, sampling-based methods are typically slow for high-dimensional inverse problems, while variational inference often lacks estimation accuracy. In this paper, we propose α-DPI, a deep learning framework that first learns an approximate posterior using α-divergence variational inference paired with a generative neural network, and then produces more accurate posterior samples through importance re-weighting of the network samples. It inherits strengths from both sampling and variational inference methods: it is fast, accurate, and scalable to high-dimensional problems. We apply our approach to two high-impact astronomical inference problems using real data: exoplanet astrometry and black hole feature extraction.

show abstract

“…Normalizing Flows (NF) (Rezende and Mohamed 2015) are used to learn transformations between data distributions with special property that their transform process is bijective and the flow model can be used in both directions. Real-NVP (Dinh, Sohl-Dickstein, and Bengio 2016) and Glow (Kingma and Dhariwal 2018) are two typical methods for NF, in which both forward and reverse processes can be processed quickly. NF is generally used to generate data from variables sampled in a specific probability distribution, such as images or audios.…”

Section: Normalizing Flowmentioning

confidence: 99%

“…For example, they estimated the multidimensional Gaussian distribution (Li et al 2021;Defard et al 2020) by calculating the mean and variance for features, or used a clustering algorithm to estimate these normal features by normal clustering (Reiss et al 2021;Roth et al 2021). Recently, some works (Rudolph, Wandt, and Rosenhahn 2021;Gudovskiy, Ishizaka, and Kozuka 2021) began to use normalizing flow (Kingma and Dhariwal 2018) to estimate distribution. Through a trainable process that maximizes the log-likelihood of normal image features, they embed normal image features into standard normal distribution and use the probability to identify and locate anomalies.…”

Section: Introductionmentioning

confidence: 99%

FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows

Yu¹,

Zheng²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Unsupervised anomaly detection and localization is crucial to the practical application when collecting and labeling sufficient anomaly data is infeasible. Most existing representation-based approaches extract normal image features with a deep convolutional neural network and characterize the corresponding distribution through non-parametric distribution estimation methods. The anomaly score is calculated by measuring the distance between the feature of the test image and the estimated distribution. However, current methods can not effectively map image features to a tractable base distribution and ignore the relationship between local and global features which are important to identify anomalies. To this end, we propose FastFlow implemented with 2D normalizing flows and use it as the probability distribution estimator. Our FastFlow can be used as a plug-in module with arbitrary deep feature extractors such as ResNet and vision transformer for unsupervised anomaly detection and localization. In training phase, FastFlow learns to transform the input visual feature into a tractable distribution and obtains the likelihood to recognize anomalies in inference phase. Extensive experimental results on the MVTec AD dataset show that FastFlow surpasses previous state-of-the-art methods in terms of accuracy and inference efficiency with various backbone networks. Our approach achieves 99.4% AUC in anomaly detection with high inference efficiency.

show abstract

Glow: Generative Flow with Invertible 1x1 Convolutions

Cited by 127 publications

References 14 publications

A New Census of the 0.2 < z < 3.0 Universe. I. The Stellar Mass Function

A New Census of the 0.2 < z < 3.0 Universe. I. The Stellar Mass Function

alpha-Deep Probabilistic Inference (alpha-DPI): efficient uncertainty quantification from exoplanet astrometry to black hole feature extraction

FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows

Contact Info

Product

Resources

About