RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior

Hu, Hong-Ye; Wu, Dian; You, Yi-Zhuang; Olshausen, Bruno; Chen, Yubei

doi:10.48550/arxiv.2010.00029

Cited by 3 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Invertible renormalization was first proposed under the name of exact holographic mapping (EHM) [24], which further leads to applications in flow-base generative models for unsupervised machine learning [17,18,20]. An invertible renormalization transformation is a bijective map R :…”

Section: Invertible Renormalization Forms a Groupmentioning

confidence: 99%

“…The related methods were developed in Refs. [17,18,20] under the name of neural-RG. A conventional choice is to take each p Z (ζ…”

Section: Rg Flow On the Field Levelmentioning

confidence: 99%

“…This idea is first proposed in Ref. [20] and further developed in later works [17,18]. The current article further develops the machine-learning RG method by combining the RG-flow model with neural ODE techniques [2], and explores its application to representation learning of sequential data.…”

Section: Introductionmentioning

confidence: 99%

“…The RG flow of field φ → ζ induces a flow of the associated probability distribution over Map(I, R n ). Under the bijective map between the original field φ and the irrelevant field ζ, the probability measure must remain invariant(18) p Φ (φ)Dφ = p Z (ζ)Dζ.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Sheshmani¹,

You²,

Fu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Following the earlier formalism of the categorical representation learning [25] by the first two authors, we discuss the construction of the "RG-flow based categorifier". Borrowing ideas from theory of renormalization group flows (RG) in quantum field theory, holographic duality, and hyperbolic geometry, and mixing them with neural ODE's, we construct a new algorithmic natural language processing (NLP) architecture, called the RG-flow categorifier or for short the RG categorifier, which is capable of data classification and generation in all layers. We apply our algorithmic platform to biomedical data sets and show its performance in the field of sequenceto-function mapping. In particular we apply the RG categorifier to particular genomic sequences of flu viruses and show how our technology is capable of extracting the information from given genomic sequences, find their hidden symmetries and dominant features, classify them and use the trained data to make stochastic prediction of new plausible generated sequences associated with new set of viruses which could avoid the human immune system.

show abstract

Section: Invertible Renormalization Forms a Groupmentioning

confidence: 99%

“…The related methods were developed in Refs. [17,18,20] under the name of neural-RG. A conventional choice is to take each p Z (ζ…”

Section: Rg Flow On the Field Levelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Sheshmani¹,

You²,

Fu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Under RG, microscopic models flow to different fixed points that distinguish different macroscopic phases of matter. More recently, the concept of RG also finds applications in machine learning and artificial intelligence (AI) [26][27][28][29][30]. Traditionally, RG is performed in the Fourier space, which involves various approximations.…”

Section: Introductionmentioning

confidence: 99%

Differentiable Programming of Isometric Tensor Networks

Geng,

Hu,

Zou

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Differentiable programming is a new programming paradigm which enables large scale optimization through automatic calculation of gradients also known as auto-differentiation. This concept emerges from deep learning, and has also been generalized to tensor network optimizations. Here, we extend the differentiable programming to tensor networks with isometic constraints with applications to multiscale entanglement renormalization ansatz (MERA) and tensor network renormalization (TNR). By introducing several gradient-based optimization methods for the isometric tensor network and comparing with Evenbly-Vidal method, we show that auto-differentiation has a better performance for both stability and accuracy. We numerically tested our methods on 1D critical quantum Ising spin chain and 2D classical Ising model. We calculate the ground state energy for the 1D quantum model and internal energy for the classical model, and scaling dimensions of scaling operators and find they all agree with the theory well.

show abstract

Probabilistic and semantic descriptions of image manifolds and their applications

Tu,

Yang,

Hartley

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

This paper begins with a description of methods for estimating probability density functions for images that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space—not every pattern of pixels is an image. It is common to say that images lie on a lower-dimensional manifold in the high-dimensional space. However, although images may lie on such lower-dimensional manifolds, it is not the case that all points on the manifold have an equal probability of being images. Images are unevenly distributed on the manifold, and our task is to devise ways to model this distribution as a probability distribution. In pursuing this goal, we consider generative models that are popular in AI and computer vision community. For our purposes, generative/probabilistic models should have the properties of (1) sample generation: it should be possible to sample from this distribution according to the modeled density function, and (2) probability computation: given a previously unseen sample from the dataset of interest, one should be able to compute the probability of the sample, at least up to a normalizing constant. To this end, we investigate the use of methods such as normalizing flow and diffusion models. We then show how semantic interpretations are used to describe points on the manifold. To achieve this, we consider an emergent language framework that makes use of variational encoders to produce a disentangled representation of points that reside on a given manifold. Trajectories between points on a manifold can then be described in terms of evolving semantic descriptions. In addition to describing the manifold in terms of density and semantic disentanglement, we also show that such probabilistic descriptions (bounded) can be used to improve semantic consistency by constructing defenses against adversarial attacks. We evaluate our methods on CelebA and point samples for likelihood estimation with improved semantic robustness and out-of-distribution detection capability, MNIST and CelebA for semantic disentanglement with explainable and editable semantic interpolation, and CelebA and Fashion-MNIST to defend against patch attacks with significantly improved classification accuracy. We also discuss the limitations of applying our likelihood estimation to 2D images in diffusion models.

show abstract

RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior

Cited by 3 publications

References 14 publications

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Differentiable Programming of Isometric Tensor Networks

Probabilistic and semantic descriptions of image manifolds and their applications

Contact Info

Product

Resources

About