Belief Propagation, Bethe Approximation and Polynomials

Abstract

Factor graphs are important models for succinctly representing probability distributions in machine learning, coding theory, and statistical physics. Several computational problems, such as computing marginals and partition functions, arise naturally when working with factor graphs. Belief propagation is a widely deployed iterative method for solving these problems. However, despite its significant empirical success, not much is known about the correctness and efficiency of belief propagation.Bethe approximation is an optimization-based framework for approximating partition functions. While it is known that the stationary points of the Bethe approximation coincide with the fixed points of belief propagation, in general, the relation between the Bethe approximation and the partition function is not well understood. It has been observed that for a few classes of factor graphs, the Bethe approximation always gives a lower bound to the partition function, which distinguishes them from the general case, where neither a lower bound, nor an upper bound holds universally. This has been rigorously proved for permanents [36,17] and for attractive graphical models [27].Here we consider bipartite normal factor graphs and show that if the local constraints satisfy a certain analytic property, the Bethe approximation is a lower bound to the partition function. We arrive at this result by viewing factor graphs through the lens of polynomials. In this process, we reformulate the Bethe approximation as a polynomial optimization problem. Our sufficient condition for the lower bound property to hold is inspired by recent developments in the theory of real stable polynomials. We believe that this way of viewing factor graphs and its connection to real stability might lead to a better understanding of belief propagation and factor graphs in general.

Short survey on stable polynomials, orientations and matchings

“…Borbényi and Csikvári [2] gave a proof using gauge transformation. The proof presented here is the first one using stable polynomials, but we remark that this result could have been easliy deduced from the paper of Straszak and Vishnoi [23] too that uses stable polynomials.…”

“…This generalization was derived by Gurvits [11] from the original paper of Schrijver [22]. Subsequently, Anari and Oveis-Gharan [1] and Straszak and Vishnoi [23] gave a proof that only relies on the theory of stable polynomials. Another possible generalization considers counting matchings of fixed size in bipartite graphs instead of perfect matchings.…”

“…Theorem 1.1 and 1.2 belong to this line of research. This area almost exclusively relies on stable polynomials [1,23] and graph covers [5,20,18,25].…”

Gauges, Loops, and Polynomials for Partition Functions of Graphical Models

“…We complement the RSP approach of [5,34,67] by merging it with the GT approach of [21,22], and thus in a sense generalizing both. Our approach consists of the following steps (see also Fig.…”

“…In this manuscript, aimed at approximating the PF, we consider Multi-Graph Models (MGMs) where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multigraph. We suggest a new methodology for analysis and computations that combines the Gauge Function (GF) technique from [21,22] with the technique developed in [34] and [5,67] based on the recent progress in the field of real stable polynomials. We show that the GF, representing a single-out term in a finite sum expression for the PF which achieves extremum at the so-called Belief-Propagation (BP) gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph.…”