The Solution of the Deep Boltzmann Machine on the Nishimori Line

Alberici, Diego; Camilli, Francesco; Contucci, Pierluigi; Mingione, Emanuele

doi:10.1007/s00220-021-04165-0

Cited by 23 publications

(19 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proof of (10) relies on the adaptive interpolation introduced in inference in order to rigorously prove replica symmetric formulas [30,36,37] (see also [32,41]). Within this technique the presence of a small perturbation in the Hamiltonian, appearing also in Proposition 2 below as , plays a fundamental regularizing role.…”

Section: Definitions and Main Resultsmentioning

confidence: 99%

An inference problem in a mismatched setting: a spin-glass model with Mattis interaction

Camilli,

Contucci,

Mingione

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The Wigner spiked model in a mismatched setting is studied with the finite temperature Statistical Mechanics approach through its representation as a Sherrington-Kirkpatrick model with added Mattis interaction. The exact solution of the model with Ising spins is rigorously proved to be given by a variational principle on two order parameters, the Parisi overlap distribution and the Mattis magnetization. The latter is identified by an ordinary variational principle and turns out to concentrate in the thermodynamic limit. The Gaussian signal distribution case is investigated and the corresponding phase diagram is identified.

show abstract

Section: Definitions and Main Resultsmentioning

confidence: 99%

An inference problem in a mismatched setting: a spin-glass model with Mattis interaction

Camilli,

Contucci,

Mingione

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the Ising case, there are conjectured formulas for the limiting free energy [18,17,51] of the bipartite SK model, although not much is known rigorously. See [4,7,35,1] for results on a generalization of the bipartite SK model, and [5,6] for its restriction to a special subset of phase space.…”

Section: Related Workmentioning

confidence: 99%

Free energy in multi-species mixed p-spin spherical models

Bates

Youngtak

2022

Electron. J. Probab.

View full text Add to dashboard Cite

“…We integrate Y which is, conditionally on (S, T ), a complex gaussian multivariate random variable, by using formula (46) and obtain that EZ(Y ) u equals…”

Section: 5mentioning

confidence: 99%

Statistical limits of dictionary learning: random matrix theory and the spectral replica method

Barbier¹,

Macris²

2021

Preprint

View full text Add to dashboard Cite

We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existing literature concerned with the low-rank (i.e., constant-rank) regime. We first consider a class of rotationally invariant matrix denoising problems whose mutual information and minimum mean-square error are computable using standard techniques from random matrix theory. Next, we analyze the more challenging models of dictionary learning. To do so we introduce a novel combination of the replica method from statistical mechanics together with random matrix theory, coined spectral replica method. It allows us to conjecture variational formulas for the mutual information between hidden representations and the noisy data as well as for the overlaps quantifying the optimal reconstruction error. The proposed methods reduce the number of degrees of freedom from Θ(N 2 ) (matrix entries) to Θ(N ) (eigenvalues or singular values), and yield Coulomb gas representations of the mutual information which are reminiscent of matrix models in physics. The main ingredients are the use of HarishChandra-Itzykson-Zuber spherical integrals combined with a new replica symmetric decoupling ansatz at the level of the probability distributions of eigenvalues (or singular values) of certain overlap matrices.

show abstract

The Solution of the Deep Boltzmann Machine on the Nishimori Line

Cited by 23 publications

References 24 publications

An inference problem in a mismatched setting: a spin-glass model with Mattis interaction

An inference problem in a mismatched setting: a spin-glass model with Mattis interaction

Free energy in multi-species mixed p-spin spherical models

Statistical limits of dictionary learning: random matrix theory and the spectral replica method

Contact Info

Product

Resources

About