Symmetries and phase diagrams with real-space mutual information neural estimation

Gökmen, Doruk Efe; Ringel, Zohar; Huber, Sebastian; Koch-Janusz, Maciej

doi:10.1103/physreve.104.064106

Cited by 12 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Information theoretical approach to RG: some research has explored the information theoretical criterion for optimal RG, indicating that RG transformation should maximize the mutual information of relevant features with the environment [1,5,[63][64][65][66] or minimize the mutual information among irrelevant features [6]. While MLRG does not conflict with these principles, it does not explicitly use them as optimization criteria.…”

Section: Summary and Discussionmentioning

confidence: 99%

Machine learning renormalization group for statistical physics

Hou,

You

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

We develop a Machine-Learning Renormalization Group (MLRG) algorithm to explore and analyze many-body lattice models in statistical physics. Using the representation learning capability of generative modeling, MLRG automatically learns the optimal renormalization group (RG) transformations from self-generated spin configurations and formulates RG equations without human supervision. The algorithm does not focus on simulating any particular lattice model but broadly explores all possible models compatible with the internal and lattice symmetries given the on-site symmetry representation. It can uncover the RG monotone that governs the RG flow, assuming a strong form of the c-theorem. This enables several downstream tasks, including unsupervised classification of phases, automatic location of phase transitions or critical points, controlled estimation of critical exponents and operator scaling dimensions. We demonstrate the MLRG method in two-dimensional lattice models with Ising symmetry and show that the algorithm correctly identifies and characterizes the Ising criticality.

show abstract

Section: Summary and Discussionmentioning

confidence: 99%

Machine learning renormalization group for statistical physics

Hou,

You

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…More recently, MI estimators based on bounds approximated by neural networks have gained interest [19,[65][66][67][68][69][70][71][72][73][74][75][76]. In particular, Belghazi et al [69] proposed a neural estimator of I(X, Y) (hereafter referred to as MINE) rewriting it as a Kullback-Leibler (KL) divergence [77], and considering its Donsker-Varadhan representation [78] (see section 2.2 for more details).…”

Section: Introductionmentioning

confidence: 99%

A robust estimator of mutual information for deep learning interpretability

Piras¹,

Peiris²,

Pontzen³

et al. 2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI (pronounced “Jimmie”), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to the choice of hyperparameters and provides the uncertainty on the MI estimate due to the finite sample size. We extensively validate GMM-MI on toy data for which the ground truth MI is known, comparing its performance against established mutual information estimators. We then demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We train deep learning models to encode high-dimensional data within a meaningful compressed (latent) representation, and use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation. Upon acceptance of this work, we will link a publicly available GitHub repository which contains GMM-MI and the code to reproduce the results of this paper.

show abstract

“…Currently, to detect phases of matter with machine learning, the widely-used approaches usually train the NN to perform a certain classification task [9,10]. In spite of their successes, they could generally face the lack of interpretability as the classes of patterns recognized by them are essentially abstract, and hence cannot assume straightforward relation to conventional notions of physics [32][33][34][35][36][37][38]. For instance, the supervised learning-with-blanking approach [9,10] investigates phase transitions by examining the intersection of NN's binary classification confidences.…”

Section: Introductionmentioning

confidence: 99%

Learning phase transitions from regression uncertainty: a new regression-based machine learning approach for automated detection of phases of matter

Guo,

2023

New J. Phys.

View full text Add to dashboard Cite

For performing regression tasks involved in various physics problems, enhancing the precision or equivalently reducing the uncertainty of regression results is undoubtedly one of the central goals. Here, somewhat surprisingly, the unfavorable regression uncertainty in performing the regression tasks of inverse statistical problems is found to contain hidden information concerning the phase transitions of the system under consideration. By utilizing this hidden information, a new unsupervised machine learning approach was developed in this work for automated detection of phases of matter, dubbed learning from regression uncertainty. This is achieved by revealing an intrinsic connection between regression uncertainty and response properties of the system, thus making the outputs of this machine learning approach directly interpretable via conventional notions of physics. It is demonstrated by identifying the critical points of the ferromagnetic Ising model and the three-state clock model, and revealing the existence of the intermediate phase in the six-state and seven-state clock models. Comparing to the widely-used classification-based approaches developed so far, although successful, their recognized classes of patterns are essentially abstract, which hinders their straightforward relation to conventional notions of physics. These challenges persist even when one employs the state-of-the-art deep neural networks that excel at classification tasks. In contrast, with the core working horse being a neural network performing regression tasks, our new approach is not only practically more efficient, but also paves the way towards intriguing possibilities for unveiling new physics via machine learning in a physically interpretable manner.

show abstract

Symmetries and phase diagrams with real-space mutual information neural estimation

Cited by 12 publications

References 33 publications

Machine learning renormalization group for statistical physics

Machine learning renormalization group for statistical physics

A robust estimator of mutual information for deep learning interpretability

Learning phase transitions from regression uncertainty: a new regression-based machine learning approach for automated detection of phases of matter

Contact Info

Product

Resources

About