Inference and mutual information on random factor graphs

Abstract

Random factor graphs provide a powerful framework for the study of inference problems such as decoding problems or the stochastic block model. Information-theoretically the key quantity of interest is the mutual information between the observed factor graph and the underlying ground truth around which the factor graph was created; in the stochastic block model, this would be the planted partition. The mutual information gauges whether and how well the ground truth can be inferred from the observable data. For a very general model of random factor graphs we verify a formula for the mutual information predicted by physics techniques. As an application we prove a conjecture about low-density generator matrix codes from [Montanari: IEEE Transactions on Information Theory 2005]. Further applications include phase transitions of the stochastic block model and the mixed k-spin model from physics.

The Ising antiferromagnet and max cut on random regular graphs

“…Moreover, results from Coja-Oghlan, Krzakala, Perkins and Zdeborová [13] imply the existence and location of a replica symmetry breaking phase transition for the Potts model on Erdős-Rényi graphs. The recent work of Coja-Oghlan, Hahn-Klimroth, Loick, Müller, Panagiotou and Pasch [15] extend these results to graphs with given degree sequences. However, the results from [15] determine the location of the replica symmetry breaking phase transition only implicitly as the solution to an infinite-dimensional variational problem.…”

“…The recent work of Coja-Oghlan, Hahn-Klimroth, Loick, Müller, Panagiotou and Pasch [15] extend these results to graphs with given degree sequences. However, the results from [15] determine the location of the replica symmetry breaking phase transition only implicitly as the solution to an infinite-dimensional variational problem. Thus, the contribution of Theorem 1.1 is the explicit analytic formula for the phase transition β * (d), which matches the combinatorially meaningful Kesten-Stigum bound [38].…”

“…Proof of Theorem 1.4. The theorem follows from Theorem 1.1, Theorem 17.1 in [15] and the fact that the disassortative stochastic block model with two communities is the planted Ising antiferromagnet.…”

“…To prove the second part of Theorem 1.1 we will put Lemma 2.6 in reverse by showing that (2.18) is violated if β > β * (d). As a stepping stone we use a variational formula for E[log Z G * ,β ] from [15]. Let P * ([−1, 1]) be the space of all probability measures on the interval [−1, 1] with mean zero.…”

“…[17,18,25,26,36,37,58]. Additionally, the Ising model is intimately related to the regular version of the disassortative stochastic block model [15], a prominent case study in Bayesian inference. 1.2.…”