Gauges, Loops, and Polynomials for Partition Functions of Graphical Models

Abstract

Graphical models (GM) represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function (PF), is the main inference challenge relevant to multiple statistical and optimization applications. The problem is #P-hard that is of an exponential complexity with respect to the number of variables. In this manuscript, aimed at approximating the PF, we consider Multi-Graph Models (MGMs) where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multigraph. We suggest a new methodology for analysis and computations that combines the Gauge Function (GF) technique from [21,22] with the technique developed in [34] and [5,67] based on the recent progress in the field of real stable polynomials. We show that the GF, representing a single-out term in a finite sum expression for the PF which achieves extremum at the so-called Belief-Propagation (BP) gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, GF can be used to recover the PF through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists in the application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination/contraction of edges resulting in MGMs on decreasing in size (number of edges) graphs with the same PF as in the original MGM. Even though the complexity of computing factors in the sequence of derived MGMs and respective GFs grow exponentially with the number of eliminated edges, polynomials associated with the new factors remain bi-stable if the original factors have this property. Moreover, we show that BP estimations in the sequence do not decrease, each low-bounding the PF.