HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference

Abstract

Many recent invertible neural architectures are based on coupling block designs where variables are divided in two subsets which serve as inputs of an easily invertible (usually affine) triangular transformation. While such a transformation is invertible, its Jacobian is very sparse and thus may lack expressiveness. This work presents a simple remedy by noting that subdivision and (affine) coupling can be repeated recursively within the resulting subsets, leading to an efficiently invertible block with dense, triangular Jacobian. By formulating our recursive coupling scheme via a hierarchical architecture, HINT allows sampling from a joint distribution p(y,x) and the corresponding posterior p(x|y) using a single invertible network. We evaluate our method on some standard data sets and benchmark its full power for density estimation and Bayesian inference on a novel data set of 2D shapes in Fourier parameterization, which enables consistent visualization of samples for different dimensionalities.

Improvement of Outpatient Service Processes: A Case Study of the University of Hong Kong-Shenzhen Hospital

“…Similarly, Behrmann et al [4] proposed invertible residual networks (i-ResNet), introducing a tractable estimation to the Jacobian log-determinant of a residual block. Other representative normalizing flow work includes hierarchical recursive coupling [23] for increasing flow expressiveness, Wavelet Flow [53] for scaling flow to ultra-high dimensional data, and Discrete Flow [48] for discrete data modeling. In this paper, we do not use normalizing flow for generative modeling, but for invertible feature transform.…”

Invertible Neural Networks in Astrophysics

“…The cINN architecture, as illustrated in Figure 1, avoids these problems [2,3,29,30,37]. It uses a di↵erent mapping system by considering the observations y in both the forward and inverse process as a condition c: f (x; c = y) = z, x = g(z; c = y) [3].…”

Self-reinforced polynomial approximation methods for concentrated probability densities

“…In this setting, the KL divergence D KL (ν X T ν U ) is approximated by the Monte Carlo method using samples drawn from ν X . The normalizing flow methods, e.g., [7,9,30,40], adopt a similar KL minimization problem, but the maps may not be triangular and are often parametrized by neural networks. By and large, the map-from-samples approach is flexible to implement, as it only requires a set of samples drawn from ν X .…”