The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models

Abstract

In this contribution we give a pedagogic introduction to the newly introduced adaptive interpolation method to prove in a simple and unified way replica formulas for Bayesian optimal inference problems. Many aspects of this method can already be explained at the level of the simple Curie-Weiss spin system. This provides a new method of solution for this model which does not appear to be known. We then generalize this analysis to a paradigmatic inference problem, namely rank-one matrix estimation, also refered to as the Wigner spike model in statistics. We give many pointers to the recent literature where the method has been succesfully applied.Keywords adaptive interpolation · Bayesian inference · replica formula · matrix estimation · Wigner spike model · Curie-Weiss model · spin systems 1 IntroductionThe replica method from statistical mechanics has been applied to Bayesian inference problems (e.g., coding, estimation) already two decades ago [1,2]. Rigorous proofs of the formulas for the mutual informations/entropies/free energies stemming from this method, have for a long time only been partial, consisting generally of one sided bounds [3,4,5,6,7,8,9,10]. It is only quite recently that there has been a surge of progress using various methods -namely spatial coupling [11,12,13,14,15], information theory [16], and rigorous versions of the cavity method [17,18,19]-to derive full proofs, but which are typically quite complicated. Recently we introduced [20] a powerful evolution of the Guerra-Toninelli [21,22,23] interpolation method -called adaptive interpolation-that allows to fully prove the replica formulas in a quite simple and unified way for Bayesian inference problems. In this contribution we give a pedagogic introduction to this new method.We first illustrate how the adaptive interpolation method allows to solve the well known Curie-Weiss model. For this model a simple version of the method contains most of the crucial ingredients. The present rigorous method of solution is new and perhaps simpler

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

“…Concentration of overlap has been shown for various Bayesian inference problems, see, e.g., [7,8,18]. These proofs can be adapted to the present case.…”

“…In this paper we focus on the mutual information of the two-group SBM with possibly asymmetric group sizes, in dense regimes where the expected degree of the nodes diverges with the total number of nodes (and is independent of the group label). We rigorously determine a single-letter variational expression for the asymptotic mutual information by means of the recently developed adaptive interpolation method [7,8].…”

Concentration of the Matrix-Valued Minimum Mean-Square Error in Optimal Bayesian Inference

“…The fluctuations of this matrix are easier to control than the ones of the overlap because L is related to the λ ngradient of the free energy, which is self-averaging by hypothesis (9). The proof is a straightforward extension to the matrix case of the one found in [23], [30] and requires no new ideas. This general result does not depend on the fact that we consider optimal Bayesian inference; it is only a consequence of the perturbation, i.e., the side information coming from the channel (6).…”

“…the product posterior measure, and g is any bounded function. This innocent-looking key identity on which relies the whole proof follows directly from Bayes' law -thus the importance of placing ourselves in the Bayesian optimal setting-, see [23], [30]. Applied to…”

Marginals of a spherical spin glass model with correlated disorder

“…More recently, there have also been a wide variety of inference problems that were successfully studied by applying its tools and heuristics [25,36]. Some of the strategies used to approach these problems include belief-propagation and approximate message passing algorithms [12,20], along with the cavity [1,9,26,27,34], interpolation [18,34], and adaptive interpolation methods [2,3]. All of these allow to establish the asymptotic logpartition function of physical and information processing systems and the performance of Bayesian estimators but rely, in most cases, on the randomness defining the model (the "quenched disorder") being a collection of i.i.d.…”